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FOREWORD 


The  Technologies  for  Skill  Acquisition  and  Retention  Technical  Area  of  the  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  performs  research 
and  development  in  education  as  part  of  its  program.  A  major  focus  of  this  research  is 
the  development  of  information  on  which  the  Department  of  the  Army  can  base 
decisions  about  its  basic  skills  education  programs.  The  work  described  in  this  report 
was  conducted  under  Project  A 794,  Education  and  Training,  as  part  of  Task  311, 
improving  Job  Skills  Education  for  Soldiers. 

This  report  describes  work  carried  out  at  the  request  of  the  Chief,  Soldier 
Education  Division,  Total  Army  Personnel  Agency,  who  requested  that  ARI  evaluate 
the  Job  Skills  Education  Program  (JSEP),  a  computer-based,  job-relevant  curriculum 
designed  to  teach  soldiers  competencies  prerequisite  to  learning  job  skills.  Researchers 
observed  and  gathered  data  on  the  program  at  seven  Army  installations  at  which  it  was 
being  pilot-tested,  and  interviewed  and  administered  questionnaires  to  program 
participants  and  administrators.  Recommendations  are  offered  for  retaining, 
improving,  and  omitting  features  of  the  program. 

This  evaluation  effort  was  supported  by  the  Soldier  Education  Division,  Total 
Army  Personnel  Agency,  Office  of  the  Deputy  Chief  of  Staff  for  Personnel.  Division 
staff  members  were  provided  with  copies  of  the  report  in  August  1988. 
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EVALUATION  OF  THE  JOB  SKILLS  EDUCATION  PROGRAM:  LEARNING 
OUTCOMES 

EXECUTIVE  SUMMARY 


Requirement: 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  was 
asked  to  provide  a  comprehensive,  independent  review  of  the  Job  Skills  Education 
Program  (JSEP).  To  accomplish  this  mission,  the  Army  Research  Institute  contracted 
with  the  American  Institutes  for  Research  (AIR)  to  evaluate  the  effects  and  efficiency  of 
JSEP,  a  program  of  computer-based  instruction  in  the  verbal  and  quantitative  skills 
identified  as  prerequisite  competencies  for  the  94  most  densely  populated  Army 
Military  Occupational  Specialties  (MOS).  The  program  is  not  intended  to  teach  job 
skills,  but  to  instruct  soldiers  in  language  and  mathematical  skills  required  to  perform 
their  jobs  successfully. 

Of  the  186  lessons  comprising  JSEP,  159  are  computerized,  on-line  instruction  and 
27  are  paper-based.  Soldiers  enrolling  in  JSEP  are  assigned  a  "prescription"  that 
includes  a  core  of  lessons  common  to  all  MOS  and  additional  lessons  aimed  at 
prerequisite  competencies  specific  to  the  soldier's  MOS. 

Procedure: 

AIR  evaluated  JSEP  between  January  and  April  1988  as  it  was  pilot-tested  at 
seven  Army  installations.  Data  were  collected  from  all  instructors  and  continuing 
education  administrators  involved  with  JSEP,  223  currently  enrolled  soldiers,  25  JSEP 
graduates,  4  soldiers  who  had  dropped  out  of  JSEP,  and  15  supervisors  of  soldiers  who 
had  participated.  Enrolled  soldiers  were  pre-  and  posttested  on  an  instrument 
developed  by  the  Fort  Lewis  Education  Center  to  predict  performance  on  the  General 
Technical  (GT)  composite  of  the  Army  Services  Vocational  Aptitude  Battery.  (Although 
GT  improvement  was  not  a  stated  goal  of  JSEP,  this  test  was  considered  a  useful 
estimate  of  the  program's  global  effects  on  language  and  quantitative  skills  relevant  to 
Army  job  performance.)  At  four  of  the  sites  (for  179  soldiers),  JSEP's  programming  was 
adapted  and  we  collected  pre-  and  posttest  information  on  each  lesson  attempted  by  a 
soldier  and  provided  lesson-level  performance  and  time-to-complete  data.  Soldiers  and 
instructors  responded  to  written  questionnaires  that  examined  their  attitudes  toward 
JSEP  and  asked  for  suggestions  for  its  continued  use  and  improvement.  Finally,  teams 
of  evaluators  spent  2  days  at  each  site  observing  JSEP  in  use  and  interviewing 
participants  in  the  program. 

Findings: 

Many  soldiers  were  already  competent  in  the  skills  of  the  JSEP  lessons  prescribed 
for  their  MOS.  Half  or  more  of  the  soldiers  passed  three  fifths  of  the  lessons  on  the 
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pretest  (without  any  instruction).  When  soldiers  failed  the  pretest  and  were  assigned 
instruction,  they  generally  mastered  the  content  of  the  lessons  they  took.  Almost  four 
fifths  of  the  on-line  computer  lessons  were  passed  by  75%  or  more  of  the  soldiers  who 
failed  the  pretest  and  then  took  the  instruction. 

When  pre-  and  posttest  performance  on  the  Fort  Lewis  test  was  expressed  in 
GT  terms,  the  133  soldiers  for  whom  data  were  complete  showed  an  average  gain  of 
3  points  immediately  following  participation  in  JSEP.  Instructors  also  forwarded  actual 
GT  scores  achieved  after  JSEP  (but  with  additional  GT  preparation)  for  about  one  third 
of  the  soldiers  (69)  following  the  evaluation.  The  average  gain  was  14  points.  The 
evaluation  concluded  that  JSEP,  with  supplementary  instruction,  was  about  as  effective 
in  raising  GT  scores  as  earlier  evaluations  had  found  Basic  Skills  Education  Program 
(BSEP)  to  be,  but  that  the  effect  was  negligible  without  additional  GT  preparation. 
Again,  while  GT  improvement  is  not  a  stated  purpose  of  JSEP,  it  was  the  primary  goal 
of  86%  of  the  participating  soldiers.  Post-JSEP  TABE  (Test  of  Adult  Basic  Education) 
scores  were  provided  by  the  instructors  for  62  of  the  soldiers.  Among  these,  reading 
improved  about  0.6  grade  levels  and  mathematics  improved  about  1.6  grade  levels,  on 
the  average.  While  there  was  no  additional  instruction  between  the  time  soldiers 
completed  JSEP  and  when  they  were  administered  the  TABE,  there  was  the  possibility 
that  unrecorded  instruction  had  occurred  between  the  time  they  were  pretested  and 
their  enrollment  in  JSEP. 

There  was  considerable  variance  in  the  time  soldiers  took  to  complete  JSEP.  The 
range  for  the  first  lesson  (la),  for  example,  was  from  8  minutes  to  3  hours  and  41 
minutes.  Half  of  the  soldiers  took  43  hours  or  less  to  complete  the  common  core  of  98 
lessons  embedded  in  every  MOS  prescription — but  the  soldiers  passed  the  pretest  for 
two  thirds  of  these  lessons  and  thus  worked  through  the  instruction  for  only  about  one 
third  of  them. 

Although  instructors  had  been  asked  to  enroll  any  eligible  soldier  during  the 
evaluation,  the  evaluation  group  had  an  average  entry  level  of  10th  grade  in  reading. 
Only  one  in  four  read  below  the  9th  grade  level,  as  measured  by  the  TABE.  The  average 
mathematics  score  was  at  the  8.7  grade  level  with  only  1  soldier  in  10  scoring  below  7th 
grade.  These  reading  levels  are  almost  as  high  as  the  terminal  educational  objectives  of 
BSEP  II.  Because  of  these  high  levels,  it  is  not  possible  to  predict  whether  JSEP  will  be 
suitable  for  the  soldiers  traditionally  served  by  Army  basic  skills  programs.  Within  the 
relatively  narrow  achievement  range  observed,  soldiers  with  lower  TABE  or  GT  scores 
passed  fewer  JSEP  lessons  on  pretest  and  failed  relatively  more  JSEP  lessons  after 
instruction.  Thus,  it  seems  probable  that  the  lower  a  soldier's  reading  and  mathematics 
achievement,  the  more  relevant  to  his  or  her  deficiencies,  but  also  the  more  difficult, 
JSEP  will  be. 

Soldiers  almost  invariably  liked  JSEP.  They  stated  that  the  computer-based 
instruction  was  enjoyable,  held  their  attention,  allowed  them  to  progress  at  their  own 
pace,  forced  them  to  persevere  in  a  task  until  they  had  mastered  it,  and  allowed  them  to 
make  mistakes  in  private.  Soldiers  said  they  enrolled  in  JSEP  to  improve  their  GT  scores 
and  generally  thought  the  program  would  help  to  do  this,  although  it  was  not  designed 
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to  improve  GT  and  the  empirical  data  indicated  that  it  did  not  do  so.  All  the  JSEP 
instructors  and  educational  administrators  who  were  interviewed  were  aware  that 
JSEP's  purpose  was  to  teach  soldiers  the  verbal  and  quantitative  skills  prerequisite  to 
MOS  performance.  However,  these  educators  said  that  neither  the  soldiers  nor  the 
commanders  who  referred  them  to  continuing  education  perceived  this  to  be  a  need, 
and  job  performance  improvement  was  not  cited  as  a  reason  for  enrolling  in  continuing 
education  by  any  of  the  soldiers  or  their  supervisors  interviewed  for  the  evaluation. 

There  were  many  reports  of  errors  in  lessons,  lesson  tests,  and  the  programming 
of  the  soldier  management  system,  but  not  all  planned  revisions  had  been  instituted  at 
the  time  of  the  evaluation.  Soldiers  and  instructors  said  consistently  that  they  wanted 
JSEP's  programming  amended  to  allow  soldiers  to  see  which  items  they  had  answered 
incorrectly  on  lesson  tests.  They  also  said  that  they  wanted  the  rigidity  or  ambiguity  of 
some  test  items  changed  and  expressed  the  opinion  that  passing  a  lesson  test  often 
depended  on  factors  other  than  a  knowledge  of  the  content.  Instructors  also  expressed  a 
strong  need  for  the  diagnostic  test,  still  in  development,  that  will  determine  which  of 
the  lessons  in  a  prescription  a  soldier  needs  to  take. 

The  major  conclusion  of  the  evaluation  was  that  JSEP  generally  teaches  what  it 
sets  out  to  teach,  as  measured  by  the  individual  lesson  tests  that  are  a  part  of  the 
program.  However,  the  soldiers  in  this  evaluation  knew  much  of  the  lesson  content 
before  JSEP  instruction,  and  mastery  of  individual  lesson  content  did  not  appear  to 
have  much  effect  on  general  verbal  and  quantitative  competencies  when  these  were 
measured  by  a  test  modeled  after  the  GT  composite.  It  was  evident  in  the  evaluation 
that  JSEP's  expressed  purpose  (improving  academic  competencies  prerequisite  to  job 
performance)  was  not  congruent  with  the  goals  of  its  potential  users  (improving  GT 
score).  The  high  entering  basic  skills  achievement  of  the  soldiers  in  this  evaluation 
prevent  us  from  saying  whether  JSEP  can  be  used  as  a  substitute  for  BSEP. 

Unless  there  is  clear  and  widely  shared  understanding  of  what  JSEP  is  supposed 
to  do,  whom  it  is  intended  to  serve,  and  how  it  is  to  be  administered,  the  chances  of  the 
program  surviving  intact  seem  small.  Without  systematic  guidance  on  JSEP's  use,  it  is 
the  opinion  of  the  evaluators  that  the  program  will  become  a  set  of  discrete  lessons 
available  through  PLATO  and  will  be  used  in  a  nonstandard  manner  to  supplement 
other  basic  skills  instruction. 

Utilization  of  Findings: 

Understanding  that  the  goals  of  Army  basic  skills  education  change  to  meet  new 
needs  and  external  conditions,  our  major  recommendation  is  that  the  Soldier  Education 
Division  reconsider  the  purposes  and  potential  users  of  JSEP.  Even  if  such 
reconsideration  did  not  lead  to  a  change  in  the  program's  stated  functions  and 
audiences,  the  exercise  could  increase  agreement  about  how  JSEP  is  to  be  used. 

We  also  recommend  that  all  lesson  and  lesson  test  revisions  be  implemented 
immediately.  The  military  and  civilian  personnel  we  interviewed  had  a  low  tolerance 
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for  errors  and  inefficiency  and  in  our  judgment  even  minor  or  moderate  problems  will 
lead  them  to  reject  JSEP  in  toto.  For  the  same  reason  we  feel  it  is  important  that  the 
Soldier  Education  Division  institute  a  maintenance  program  that  will  update  JSEP  to 
prevent  obsolescence.  Great  care  was  taken  in  JSEP's  design  to  select  competencies  that 
related  to  Armv  duties,  and  to  use  military  language  and  situations  in  teaching  these 
competencies.  The  same  attention  should  be  paid  to  "greening"  JSEP's  operations — 
reflecting  Armv  values  by  making  the  program  error-free,  efficient,  and  up-to-date,  and 
by  showing  explicitly  how  it  relates  to  the  mission  of  the  Army  and  the  individual 
soldier. 

The  lesson  pretest  option,  introduced  for  the  evaluation,  should  be  retained  and 
used  to  exempt  soldiers  from  instruction  in  skills  they  already  have,  at  least  until  the 
diagnostic  test  intended  to  serve  this  purpose  has  been  completed  and  validated.  The 
programming  for  JSEP  should  be  modified  to  allow  soldiers  to  see  which  lesson  test 
items  they  have  answered  incorrectly,  and  additional  test  items  should  be  developed  so 
that  the  same  questions  are  not  presented  to  soldiers  repeatedly. 

We  also  recommend  that  the  Soldier  Education  Division  implement  some  form  of 
monitoring  system  to  examine  the  effects  and  efficiency  of  JSEP  if  and  when  it  is  in  final 
programmatic  form.  Such  action  would  provide  the  program  with  a  summative 
evaluation  and  would  help  to  keep  it  contemporary  and  relevant  to  Armv  needs. 
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EVALUATION  OF  THE  JOB  SKILLS  EDUCATION  PROGRAM: 

LEARNING  OUTCOMES 

Chapter  1.  Development  of  JSEP 


Educational  efforts  of  the  size  and  complexity  of  the  U.  S.  Army’s  Job  Skills 
Education  Program  (JSEP)  do  not  spring  into  existence  overnight.  This  chapter  describes 
briefly  the  development  and  current  structure  of  JSEP  as  these  provide  the  context  for 
the  present  evaluation. 


Events  Leading  to  JSEP 

Although  JSEP  began  officially  in  October  1982  with  an  award  through  the  U.S. 

Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  to  Florida  State 
University’s  Center  for  Educational  Technology,  the  program’s  roots  go  back  at  least  as 
far  as  January  1980.  At  that  time  the  Department  of  the  Army  Adjutant  General- 
Education  (DAAG-ED)  issued  a  lerter  on  the  topic  of  "Implementation  of  Army  Continuing 
Education  Policy  and  Recommendation  Plan,"  which  established  a  framework  for  improving 
the  Army  Continuing  Education  System.  Following  this,  TRADOC  Regulation  621-1,  Basic 
Skills  Education  Program  Curriculum  Development  Project,  set  forth  the  policies  and 
procedures  for  redesigning  the  Army’s  Basic  Skills  Education  Program  (BSEP)  in 
accordance  with  that  plan  (Hahn,  1985a).  The  original  name  for  this  program  was 
Functional  BSEP  (FBSEP),  but  it  was  subsequently  changed  to  Job  Skills  Education 
Program  (JSEP)  to  emphasize  the  program’s  intended  job  relatedness.  A  major  component 
in  the  BSEP  curriculum  development  project  was  the  identification  of  baseline  skill 
requirements  for  soldiers  at  Skill  Levels  1  and  2  (those  holding  the  rank  of  El  through 
E5)  to  be  successful  in  their  Military  Occupational  Specialties  (MOSs).  Once  these 
requisite  skills  were  identified,  the  plan  called  for  developing  training  materials  and 
measurement  instruments  to  ensure  that  the  skills  were  mastered. 

This  initial  MOS  Baseline  Skills  Project  was  carried  out  by  the  RCA  Services 
Company  for  TRADOC  (Training  and  Doctrine  Command).  RCA  examined  the  job  tasks 
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associated  with  the  94  most  densely  populated  MOSs  and  those  listed  in  the  Soldier’s 
Manual  of  Common  Tasks.  RCA  then  developed  descriptors  that  minutely  covered  each 
step  in  performing  these  job  tasks.  The  descriptors  were  analyzed  qualitatively  to  develop 
a  set  of  basic  competencies  that  a  soldier  would  need  in  order  to  learn  and  perform  the 
target  job  tasks.  From  an  initial  18-category  taxonomy  of  prerequisite  competencies,  a 
final  taxonomy  of  approximately  200  was  developed. 

RCA  then  constructed  paper-and-pencii,  multiple  choice  tests  for  the  128 
competencies  for  which  such  tests  were  deemed  suitable.  The  tests  were  to  be  used  to 
diagnose  individual  soldier  inadequacies  so  that  each  soldier  would  be  assigned  instruction 
in  only  those  competencies  in  which  a  weakness  had  been  identified  (Hahn,  1985b).  These 
multiple-choice  tests  were  validated  against  another  set  of  constructed-response  tests 
measuring  proficiency  in  the  same  competencies. 

Since  it  would  be  impractical  to  administer  128  or  more  diagnostic  tests  of  10  to  15 
items  each  to  every  soldier  who  was  eligible  for  training,  RCA  produced  a  pair  of  30-item 
Locator  Tests.  One  test  each  was  prepared  in  verbal  skills  and  mathematics.  The  Locator 
Tests  were  used  to  predict  whether  a  soldier  would  pass  each  of  the  available  competency 
tests.  Theoretically,  with  the  knowledge  of  a  soldier’s  MOS  and  the  use  of  a  limited 
number  of  diagnostic  tests,  it  would  be  possible  to  prescribe  an  instructional  sequence 
that  included  only  the  competencies  in  which  the  individual  was  weak. 

Continuation  Into  JSEP 

Although  the  final  RCA  products  were  nut  available  to  Florida  State  University  when 
it  began  developing  JSEP  under  the  technical  guidance  of  the  U.S.  Army  Research 
Institute,  and  although  Florida  State  University  was  not  contractually  bound  to  use  the 
results  of  the  RCA  project,  JSEP  is  to  a  large  extent  a  logical  continuation  of  the  MOS 
Baseline  Skills  Project.  Based  on  the  reanalysis  of  some  data,  Florida  State  University 
redefined  the  taxonomy  of  prerequisite  competencies  and  modified  the  match  between  some 
tests  and  the  competencies.  Florida  State  University  then  set  about  to  develop 
instructional  materials  that  would  teach  this  modified  taxonomy  of  prerequisite 
competencies.  About  half  of  the  resulting  lessons  were  planned  to  be  on  computer, 
although  159  of  the  current  186  lessons  (85.5%)  are  written  entirely  or  partially  as 
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computer-based  instruction1.  (The  computerized  format  was  considered  inappropriate  for 
some  prerequisite  competencies,  and  these  are  presented  as  paper-based  lessons.) 

Florida  State  University  and  the  Army  Research  Institute  continue  to  field-test  JSEP  at 
the  time  of  this  report’s  writing.  Soldiers  at  seven  Army  installations  are  working 
through  JSEP  lessons  to  evaluate  revisions  and  editorial  changes  and  to  generate  the  data 
needed  to  produce  diagnostic  tests  that  will  direct  future  learners  to  the  appropriate 
lessons. 

Reasons  for  JSEP 

Several  factors  contributed  to  JSEP’s  being  developed  in  the  first  place,  and  to  the 
differences  between  JSEP  other  Army  basic  skills  education  programs  (Farr,  1986).  One  of 
these  was  a  recommendation  from  the  General  Accounting  Office  (GAO)  that  basic  skills 
programs  be  standardized  across  Army  installations  to  ensure  quality  and  continuity  for 
soldiers  transferred  from  one  post  to  another.  This  was  complemented  by  a  congressional 
mandate  that  any  education  offered  during  duty  hours  be  job-related.  Finally,  the  Army 
Science  Board  had  recommended  that  educational  programs  take  advantage  of  new 
instructional  technology.  JSEP  has  several  characteristics  that  respond  directly  to  these 
influences: 

•  JSEP  is  based  upon  the  verbal  and  quantitative  competencies  identified  as 
prerequisites  for  the  most  widely  filled  Army  MOSs.  Converting  Julian  dates  to 
Gregorian  dates,  for  example,  is  considered  a  prerequisite  quantitative 
competency  for  MOS  55B,  Ammunition  Specialist.  JSEP  is  not  really  a  "basic 
skills  program"  if  such  programs  are  defined  as  teaching  the  entire  body  of 
skills  and  knowledge  comprising  an  academic  area.  Nor  is  JSEP  intended  to 
remedy  extensive  deficiencies;  it  will  not  teach  reading  to  a  functionally 
illiterate  adult.  It  is  designed,  however,  to  teach  specific  reading  skills  called 
on  in  the  performance  of  common  MOSs,  such  as  locating  documents  by  their 
code  and  title  or  understanding  communication  and  navigation  words  common  to 
the  Army. 


^able  A.1,  Appendix  A,  shows  186  lessons  written,  1  lesson  not  available,  and  1 
lesson  not  written,  for  a  total  of  188. 
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•  On  the  other  hand,  JSEP  is  not  job  training.  It  may  teach  the  reading  and 
mathematical  skills  needed  to  use  a  tank  repair  manual,  but  it  will  not  train  a 
soldier  to  repair  a  tank.  While  the  instruction  was  "greened,"  or  made  Army 
relevant  where  possible,  it  still  largely  teaches  verbal  and  mathematical  skills, 
not  job  skills. 

•  JSEP  is  computer-based  instruction.  It  includes  lessons  and  instructional 
management  software  that  determines  what  instructional  materials  are  presented 
to  the  soldier.  A  complete  "prescription"  of  lessons  is  selected  on  the  basis  of 
the  soldier’s  MOS.  Progress  through  this  prescription  is  controlled  by  internal 
constraints  (some  lessons  are  prerequisites  to  others)  and  by  the  soldier’s 
performance.  Repeated  inability  to  master  a  lesson  will  shunt  a  soldier  out  of 
the  system  for  instructor  assistance.  Otherwise  the  system  is  designed  to 
require  little  instructor  intervention,  which  reduces  variability  across  sites.  The 
management  system  makes  it  theoretically  possible  for  a  soldier  to  complete  one 
lesson  at  a  post  in  Georgia  and  begin  the  next  at  a  new  post  in  Germany. 


Description  of  JSEP 

JSEP  includes  two  kinds  of  lessons:  content  lessons  and  learning  strategy  lessons. 

It  also  has  both  computerized  and  paper  formats.  The  following  paragraphs  are  intended 
as  a  brief  overview  for  readers  who  are  not  familiar  with  JSEP.  For  a  comprehensive 
written  description  of  the  program,  please  see  the  JSEP  Instructor’s  Manual  (Schroedl  and 
Branson,  1987). 

Content  Lessons 

By  far  the  largest  part  of  the  program,  the  content  consists  of  186  prerequisite 
competency  lessons  distributed  across  33  series.2  Table  A.1,  printed  in  Appendix  A 
because  of  its  length,  lists  these  in  abbreviated  form.  The  First  series,  for  example,  is  on 


descriptions  of  JSEP  reflect  the  program  during  the  evaluation  period  of  January- 
April  1988.  Not  all  lessons  were  in  their  final  form  at  that  time. 
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Numbering  and  Counting  arid  includes  lessons  on  such  prerequisite  competencies  as 
matching  numerals  with  their  names  and  rounding  whole  and  decimal  numbers. 

Instructionally,  there  are  two  kinds  of  content  lessons  that  may  be  used  to  teach  a 
prerequisite  competency.  Diagnostic  Review  Lessons  assume  the  soldier  once  knew  the 
material  but  cannot  recall  it.  These  lessons  are  relatively  short;  as  their  name  implies, 
they  are  designed  to  provide  a  review  of  content  and  to  diagnose  whether  further 
instruction  is  needed.  All  computer-instructed  competencies  include  Diagnostic  Review 
Lessons. 

Many  computer-based  competencies  also  include  Skill  Development  Lessons.  These 
longer  lessons  assume  that  the  material  is  new  to  the  learner.  They  are  more  detailed 
and  tutorial  in  form  than  their  Diagnostic  Review  counterparts.  Most,  but  not  all, 
prerequisite  competencies  have  Skill  Development  Lessons. 

The  standard  instructional  format  has  a  soldier  begin  with  the  Diagnostic  Review 
Lesson  of  a  prerequisite  competency  assigned  on  the  basis  of  his  or  her  primary  or 
secondary  MOS3.  The  soldier  takes  a  short,  computer-administered  test  at  the  end  of  the 
Diagnostic  Review  Lesson.  Soldiers  who  pass  this  test  are  routed  to  choose  another 
lesson  in  their  prescription.  Those  who  fail  are  moved  into  the  prerequisite  competency’s 
Skill  Development  Lesson.  If  soldiers  fail  the  exit  test  after  two  attempts  at  this  longer 
lesson,  or  if  there  is  no  Skill  Development  Lesson  for  the  competency,  the  instructional 
management  system  refers  them  to  the  human  instructor  for  assistance. 

The  number  of  test  items  and  the  passing  score  for  each  lesson  are  shown  on  Table 
A.1.  The  passing  scores  and  usually  the  test  items  are  the  same  for  both  Diagnostic 
Review  and  Skill  Development  Lessons.  The  modal  test  contains  10  items  and  has  a 
passing  score  of  80  percent  correct. 

A  third  piece  of  information  displayed  on  Table  A.1  is  each  lesson’s  revision  status 
at  the  time  of  the  evaluation.  A  "Y"  showing  that  the  revision  process  was  completed 
indicates  tha.  the  lesson  had  been  attempted  by  10  or  more  soldiers,  reviewed  by  the 
Army  Research  Institute  and  participants  at  the  Army  installations  field-testing  JSEP, 

3The  procedures  in  place  during  the  evaluation  differed  somewhat  from  this  description. 
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reviewed  and  revised  (if  necessary)  by  Florida  State  University,  and  returned  to  the  Army 
Research  Institute.  (This  description  collapses  15  steps  between  the  initial  draft  and  the 
final  lesson.)  For  lessons  marked  with  an  "N,"  the  process  had  been  begun  but  was  not 
completed  by  January  1988.  Either  one  or  both  lessons  was  not  in  its  final  form  for  95 
(51%)  of  the  prerequisite  competencies  at  the  time  of  the  evaluation. 

Kinds  of  Formats 

Both  Diagnostic  Review  and  Skill  Development  Lessons  can  be  presented  in  a  mix  of 
computerized  and  paper  formats.  Most  lessons  are  completely  computerized.  Some  (27,  or 
14.5  percent)  are  completely  paper  based.  A  few  (5,  or  2.7  percent)  have  some  mix  of 
formats.  They  may  include  a  computerized  lesson  with  a  paper  based  test  or  a 
computerized  Diagnostic  Review  Lesson  and  a  paper  Skill  Development  Lesson.  Finally, 
computerized  lessons  may  assign  paper  supplementary  materials  to  be  used  with  on-line 
computer  instruction  and  testing.  Lesson  28d,  which  requires  using  information  from 
tables  or  charts  to  locate  a  malfunction,  is  an  example  of  the  mixed  paper  and  on-line 
format. 

Learning  Strategy  Lessons 

There  are  five  modules  that  are  concerned  with  learning  strategies  rather  than  MOS- 
related  content  Their  function  is  to  teach  soldiers  how  to  leant  The  Time  Management 
module  is  assigned  by  the  JSEP  management  system  to  every  soldier  as  he  or  she  begins  a 
JSEP  prescription.  The  remaining  four  learning  strategies  will  be  assigned  automatically 
once  a  soldier  fails  a  given  number  of  tests  or  may  be  assigned  by  the  instructor.  These 
modules  are  Reading  Strategies,  Problem  Solving  (mathematics),  Test-Taking,  and 
Motivational  Skills.  The  JSEP  management  system,  for  example,  assigns  the  Reading 
Strategies  Module  when  a  soldier  fails  five  tests  that  have  a  predetermined  verbal  content. 

PLATO  and  MicroTICOT  Systems 

One  final  distinction  must  be  made  in  order  to  understand  the  JSEP  evaluation.  This 
is  the  difference  between  the  PLATO  and  MicroTTCCIT  systems  through  which  JSEP  was 
offered  during  the  evaluation.  PLATO  is  the  computer  system  used  at  most  Army 
Education  Centers.  Sites  using  the  PLATO  system  had  computer  terminals  linked  to  a 
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central  computer  at  one  of  two  sites  (Fort  Leavenworth  or  Fc  rt  udvoir).  The 
Micro TICCIT  system  was  composed  of  IBM  personal  computers  operating  from  a 
minicomputer  at  the  individual  site.  Two  of  the  seven  pilot  sites  used  the  MicroTICCIT 
system.  Because  PLATO  was  used  by  a  larger  sample  of  soldiers  and  because  it  was 
possible  to  collect  data  on  soldiers’  lesson-level  performance  in  the  PLATO  system,  the 
PLATO-offered  version  of  JSEP  has  received  the  most  attention  in  this  report. 

This  chapter  has  sketched  the  conditions  under  which  JSEP  developed  and  the  major 
characteristics  of  the  program.  The  program  and  its  context  in  turn  determined  the 
design  of  the  evaluation,  which  is  presented  in  the  next  chapter. 


Chapter  2.  Conducting  the  Evaluation 


Developing  a  Useful  Design 

A  basic  tenet  of  AIR’s  approach  to  program  evaluation  is  that  each  evaluation  should 
be  conceptualized  as  a  step  in  building  a  solution  to  a  problem,  rather  than  as  an  isolated 
measure  of  how  much  a  problem  has  been  reduced  (AIR,  1986).  Viewing  evaluation  as  a 
method  for  problem  solution  leads  to  studies  that  emphasize  understanding  the  causes, 
contexts  and  effects  associated  with  programs  so  that  each  subsequent  program  attempt, 
as  it  is  informed  by  evaluation  findings,  is  more  effective  and  efficient.  Evaluations  that 
focus  on  problem  reduction,  however,  are  strictly  judgmental  rather  than  explanatory. 

Each  problem  reduction  evaluation  is  undertaken  in  an  unconnected,  yes-no  framework 
that  may  tell  whether  the  program  works,  but  that  gives  very  little  information  to  guide 
future  improvements  or  suggest  alternative  treatments. 

JSEP  as  a  Solution  to  a  Problem 

JSEP  is  a  good  example  with  which  to  contrast  the  problem  solution  and  problem 
reduction  approaches.  JSEP  was  designed  on  the  assumption  that  limited  proficiency  in 
basic  academic  skills  identified  as  prerequisite  competencies  prevents  soldiers  from 
successfully  learning  and  performing  their  MOS  job  tasks.  A  problem  reduction  approach 
to  evaluation  would  seek  to  determine  whether  remedial  basic  skills  courses  led  to 
improved  MOS  task  performance.  This  seems  a  plausible  and  useful  question,  but  it  has 
two  basic  flaws.  The  first  is  that  it  assumes  that  basic  skill  proficiency  is  enough  in 
itself  to  improve  military  job  performance  --  an  assumption  that  should  be  tested,  not 
accepted  automatically.  The  front-end  task  analysis  that  was  accomplished  before  JSEP 
was  initiated  did  not  explore  empirically  the  relationship  between  basic  skills  mastery  and 
competency  in  military  jobs.  There  is  a  real  possibility  that  learning  the  prerequisite 
competencies  for  an  MOS  will  not  of  itself  bring  about  better  MOS  performance. 
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The  second  flaw  is  that  the  question,  "Was  the  program  enough?"  is  of  little  help  to 
the  policy  maker  or  program  planner  if  the  answer  is  "No."  It  provides  no  insights  about 
the  myriad  of  other  factors  that  could  affect  MOS  performance  and  does  not  give  the 
planner  any  place  to  go  except  "back  to  the  drawing  board." 

An  evaluation  based  upon  the  solution  building  approach,  in  contrast,  would 
conceptually  break  JSEP  into  its  logical  steps  and  examine  each  one  in  tum.  It  would 
accept  the  initial  assumption  that  basic  academic  competencies  are  a  necessary  prerequisite 
to  learning  and  performing  MOSs,  but  not  the  subsequent  assumption  that  they  are 
sufficient  on  their  own  to  guarantee  success.  In  AIR’s  solution  building  approach,  we 
assume  that  a  given  program  treats  but  one  component  of  the  problem  it  addresses.  For 
the  JSEP  evaluation  this  led  to  the  primary  question  of  whether  participation  in  remedial 
education  in  basic  academic  competencies  (JSEP)  improved  performance  in  these 
competencies.  This  meant  that  the  JSEP  lessons  and  prescriptions  would  be  evaluated  on 
the  specific  outcomes  they  were  intended  to  produce,  namely  an  increased  proficiency  in 
the  prerequisite  competencies  it  taught,  and  not  on  other  desirable  outcomes,  such  as 
improved  MOS  task  performance,  that  could  be  dependent  upon  factors  in  addition  to 
JSEP. 

Incremental  Knowledge  Building 

The  preceding  argument  has  methodological  strengths,  but  it  will  not  fully  meet 
policy  makers’  and  administrators’  information  needs  unless  evaluation  is  seen  as  a 
continuing  process  and  not  a  single  study.  The  Army  does  not  support  JSEP  merely  to 
increase  soldiers’  proficiency  in  selected  areas  of  language  and  mathematics.  The  ultimate 
goal  is  to  ensure  competent  job  performance.  It  seems  plausible  that  other  program 
components  in  addition  to  JSEP  will  play  a  role  in  reaching  this  goal:  the  nature  of  the 
population  pool  from  which  recruits  are  drawn,  the  instructional  technology  used  in  MOS 
training,  changes  in  the  nature  of  Army  jobs,  to  name  but  a  few  possibilities.  Thus,  an 
ongoing  evaluative  process  might  begin  by  determining  whether  JSEP  was  improving 
prerequisite  competencies  and  by  pinpointing  weaknesses  that  kept  it  from  doing  so,  but  it 
would  not  stop  there.  It  would  also  help  identify  the  next  important  steps  to  add  to  the 
solution  process.  For  maximum  effect  a  program  of  evaluation  should  be  incremental  in 
nature. 
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The  first  step  in  an  incremental  evaluation  is  to  determine  the  extent  to  which  a 
program  has  achieved  its  most  direct  and  immediate  objectives.  The  next  step  is 
contingent  upon  the  outcomes  of  the  first.  If  the  program  is  not  doing  what  it  was 
intended  to  do,  the  evaluation  moves  backwards  to  determine  where  the  course  is 
breaking  down  and  what  can  be  done  to  make  it  operate  smoothly.  If  the  immediate 
objectives  are  being  met,  the  evaluation  moves  forward  to  examine  the  effects  of  these 
outcomes  on  the  more  distal  program  goals.  This  progression  for  JSEP  might  begin  by 
determining  whether  instruction  improved  soldiers’  performance  on  the  program’s  lesson 
tests  of  specific  prerequisite  competencies;  then  move  to  examining  whether  program 
graduates  could  use  these  competencies  in  a  more  realistic  setting;  then  examine  whether 
mastery  of  these  competencies  was  related  to  quality  of  MOS  performance;  and  so  on, 
testing  each  link  in  the  causal  chain  from  prerequisite  competency  to  MOS. 

Incremental  Model  for  JSEP 

In  our  previous  evaluations  of  Army  Continuing  Education  programs,  AIR  developed  a 
conceptual  model  for  a  quality  control  system  that  would  fuel  an  incremental  evaluation  or 
monitoring  process  for  the  Army’s  remedial  education  efforts.  The  major  functions  are 
shown  in  Figure  2.1,  and  the  model  was  intended  to  serve  as  the  guide  for  this  evaluation. 
Following  this  model  would  have  "piloted"  a  system  for  data  collection  and  use  that  we 
had  recommended  the  Army  Continuing  Education  System  adopt  (AIR,  1984).  It  also  would 
have  established  the  groundwork  for  an  incremental  evaluation  to  determine  JSEP’s 
eventual  effects  on  MOS  performance. 

Unfortunately,  conducting  the  JSEP  evaluation  according  to  the  proposed  quality 
control  system  model  was  not  possible.  Doing  so  required  several  program  conditions  that 
could  not  be  met  within  the  time  available  for  the  evaluation. 

limitations  on  the  Evaluation 

Applying  the  quality  control  system  required  that  all  JSEP  lessons  and  lesson  tests, 
the  computerized  instructional  management  system,  and  diagnostic  or  placement  measures 
be  completed  and  in  use  at  the  beginning  of  the  evaluation,  which  was  originally  proposed 
to  extend  from  1  April  1987  through  31  March  1988.  The  JSEP  evaluation  actually  began 
data  collection  on  19  January  1988.  At  that  time  about  30  percent  of  the  lessons  had  not 
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Figure  2.1.  Functions  of  Proposed  Quality  Control  System 


undergone  the  entire  review  and  revision  process,  several  functions  of  the  management 
system  had  not  been  introduced,  and  there  were  no  diagnostic  procedures.  In  fact, 
diagnostic  test  validation  activities  began  simultaneously  with  the  program  evaluation. 

These  problems  led  to  a  secondary  constraint  upon  the  evaluation.  Because  pilot  sites  had 
been  called  upon  for  so  much  assistance  in  developing  JSEP,  both  the  evaluators  and  their 
clients  recognized  a  need  to  limit  the  evaluation  burden.  The  data  collection  period  was 
thus  reduced  to  90  days. 

These  constraints  limited  the  extent  to  which  the  quality  control  system  model  could 
drive  the  evaluation.  It  was  possible  to  collect  information  about  whether  courses  were 
implemented  as  planned  and  how  much  was  learned  (see  figure  2.1).  However,  the 
question  of  the  extent  to  which  the  target  population  was  being  served  was  moot. 

Because  JSEP  was  still  in  a  developmental  stage,  the  number  of  program  participants  was 
limited  to  the  number  of  computer  work  stations  at  each  pilot  site.  Time  constraints 
made  it  impossible  to  look  at  another  aspect  in  quality  control:  whether  JSEP  satisfied 
external  criteria  such  as  enhancing  occupational  proficiency  or  career  progression.  These 
outcomes  could  not  be  reasonably  expected  in  a  90-day  period. 

Logical  and  logistical  problems  prevented  us  from  clearly  determining  how  much  basic 
skill  competencies  had  been  improved,  or  recommending  competency  standards  or  structural 
program  changes.  While  there  were  short  (10  item  or  so)  tests  for  each  JSEP  lesson, 
there  was  no  general  measure  of  all  the  prerequisite  competencies.  Nor  was  there  a  body 
of  research  clearly  linking  mastery  of  the  JSEP-identified  competencies  with  performance 
on  Army  ability  tests  (such  as  the  General  Technical  composite,  the  GT)  or  tests  of  basic 
skills  (such  as  the  TABE,  Tests  of  Adult  Basic  Education)  that  had  a  role  in  the  career 
and  educational  futures  of  JSEP-eligible  soldiers.  Without  this  empirical  evidence  we  were 
limited  in  how  much  we  could  say  about  JSEP’s  effects  on  basic  skill  competencies  in 
general,  and  it  would  have  been  irresponsible  to  even  attempt  recommending  competency 
standards  for  the  program.  These  constraints  limited  in  their  turn  the  judgments  we 
could  make  about  the  efficiency  of  the  program’s  overall  structure. 

Revisions  in  evaluation  design.  Recognizing  the  impact  that  these  situational 
constraints  had  upon  the  proposed  evaluation  and  other  activities,  the  Chief  of  the  Soldier 
Education  Division,  Office  of  the  Deputy  Chief  of  Staff  for  Personnel  (ODCSPER),  called  a 
meeting  of  contractors,  pilot  site  staff  persons,  and  staff  of  the  Education  Division  and 
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the  Army  Research  Institute  on  5  -  7  December  1987.  The  group  discussed  upcoming  JSEP 
demonstration,  evaluation  and  test  validation  activities.  AIR  presented  a  revised 
evaluation  plan  at  that  meeting,  one  which  did  not  change  the  original  purposes  of  the 
evaluation  but  which  adapted  the  scope  of  activities  to  the  constraints  of  limited  time  and 
information.  This  plan  was  agreed  to  by  the  meeting  participants  and  accepted  by  the 
.Army  Research  Institute,  which  had  contracted  for  the  evaluation  with  AIR.  Because  the 
evaluation,  as  it  was  conducted,  differed  somewhat  from  that  originally  proposed,  a  brief 
summary  is  offered  here. 


Components  of  the  Evaluation  Design 


Purpose  and  Questions 

The  evaluation’s  purpose  was  to  assist  the  Army  Research  Institute  in  providing  a 
comprehensive,  independent  review  of  the  effectiveness  of  JSEP.  In  order  to  do  so,  the 
evaluation  addressed  several  major  questions: 

•  Does  JSEP  teach  what  it  sets  out  to  teach? 

Does  soldiers’  performance  on  tests  of  specific  prerequisite  competencies  change 
following  instruction  in  those  competencies?  What  proportion  of  soldiers  master 
the  competencies  following  instruction? 

•  What  effect  does  JSEP  have  on  academic  requirements  for  iob  performance?  Do 
soldiers  master  the  lessons  comprising  a  complete  MOS-related  prescription? 
Does  performance  on  a  global  measure  of  language  and  mathematics  change 
following  participation  in  JSEP?  Do  supervisors  report  changes  in  soldiers’  MOS 
performance? 

•  Does  JSEP  work  as  it  was  intended  to? 

What  numbers  and  types  of  soldiers  use  JSEP  successfully?  How  long  does  it 
take  to  complete  a  JSEP  prescription?  How  do  soldiers,  supervisors,  and 
instructors  rate  JSEP’s  effectiveness  and  efficiency? 
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•  What  principles  of  instructional  design  do  the  JSEP  lessons  and  support 

materials  incorporate?  Do  the  lessons  demonstrate  clear  purpose,  organization, 
and  presentation  of  content?  ?v.  textual  support  materials,  such  as  the  JSEP 
Instructor’s  Manual,  meet  the  purposes  for  which  they  were  written? 

Evaluation  Audiences 

AIR  has  conducted  this  evaluation  for  the  Army  Research  Institute,  which  is 
responsible  for  identifying  audiences  and  authorizing,  or  conducting,  the  distribution  of 
evaluation  findings.  Thus,  staff  of  the  Army  Research  Institute  were  asked  to  review 
plans  for  data  analysis  and  report  design  and  to  suggest  ways  in  which  the  information 
could  be  presented  in  the  most  useful  manner. 


Structure  and  Activities  of  the  Evaluation 

The  evaluation  used  a  single-group,  pretest-posttest  design.  Information  was 
collected  from  soldiers  before,  during,  and  at  the  end  of  their  participation  in  JSEP.  This 
was  an  acceptable  approach  since  we  were  not  concerned  with  comparing  JSEP  to  another 
program,  and  we  felt  reasonably  certain  that  any  change  in  skills  or  attitudes  could  be 
attributed  to  JSEP  rather  than  external  conditions. 

Conditions  of  the  evaluation.  A  logistical  constraint  affecting  the  evaluation  was  the 
desire  to  place  as  little  burden  as  possible  upon  the  voluntarily  participating  sites.  Hence, 
existing  data  were  used  wherever  possible.  The  data  collection  period  was  limited  to  90 
days  between  19  January  and  15  April  1988.  Sites  later  were  given  the  option  of 
extending  the  final  date  to  6  May  in  order  to  provide  complete  information  for  soldiers 
who  had  enrolled  near  the  end  of  the  evaluation  period. 

The  JSEP  instructors  were  told  to  literally  "go  by  the  book"  during  the  evaluation 
period,  to  follow  the  procedures  outlined  in  the  JSEP  Instructor’s  Manual  and  intervene  as 
little  as  possible  in  soldiers’  learning.  They  were  asked  to  assign  soldiers  to  existing 
JSEP  prescriptions  and  not  to  design  prescriptions  on  their  own.  Instructors  also  were 
asked  not  to  use  any  supplementary  materials  or  instruction  until  soldiers  had  exited  JSEP, 
on  the  argument  that  the  evaluation  was  intended  to  test  "pure  JSEP."  Finally, 
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instructors  were  directed  to  enroll  eligible  soldiers  regardless  of  their  reading  and 
mathematics  achievement  levels.  Eligibility  was  operationally  defined  as  referral  of  a  Skill 
Level  1  or  2  soldier  due  to  a  lack  of  the  basic  skills  to  successfully  carry  out  his  or  her 
MOS  or  to  meet  job  and  reenlistment  qualification  criteria  on  other  Army  tests. 

The  JSEP  management  system  for  the  sites  using  the  PLATO  system  was  modified 
during  the  evaluation  to  pretest  soldiers  before  they  attempted  each  lesson.  Soldiers 
passing  a  pretest  were  routed  out  of  the  lesson  and  told  to  choose  another  from  their 
prescription.  Those  failing  the  pretest  were  guided  into  the  lesson’s  instructional  content. 
Adding  this  pretest  option  meant  that  participants  were  given  only  two  attempts  (rather 
than  the  standard  three)  at  the  Skill  Development  Lesson,  if  one  existed  for  the 
prerequisite  competency.  The  MicroTICCIT  system  was  not  modified  to  administer  a 
pretest,  and  so  the  data  collected  from  PLATO  and  MicroTICCIT  sites  were  not 
comparable.  Prerequisite  competency  performance  data  presented  in  this  report  are  limited 
to  the  PLATO  sites. 

Participants.  Seven  JSEP  programs  were  included  in  the  evaluation.  Two  of  these 
sites.  Fort  Lewis  and  Fort  Riley,  used  the  MicroTTCCIT  system.  Fort  Lewis  had  worked 
with  JSEP  since  September  1985,  Fort  Riley  since  July  1985.  The  five  remaining  sites 
used  the  PLATO  system.  Of  these,  Fort  Leonard  Wood,  beginning  with  JSEP  in  July  1985, 
and  Fort  Sill  (since  August  1986),  were  the  original  development  sites.  Fort  Bliss  and 
Fort  Jackson  joined  the  pilot  group  in  the  summer  of  1987,  and  Walter  Reed  Army  Medical 
Center  began  using  JSEP  during  the  evaluation  period.  Participation  was  formally 
requested  by  the  Chief  of  the  Soldier  Education  Division,  and  agreed  to  by  all  sites.  All 
soldiers  enrolling  in  JSEP  between  19  January  and  15  April  1988  were  included  in  the 
evaluation  data  base.  Their  characteristics  are  discussed  in  Chapter  4. 

Other  participants  included  supervisors  of  current  or  past  JSEP  students,  a  sample  of 
soldiers  who  had  completed  JSEP  before  the  evaluation,  and  local  Education  Center  staff 
and  JSEP  instructors. 

Sources  of  data.  There  were  three  major  kinds  of  information:  that  collected 
automatically  as  a  routine  part  of  JSEP’s  operations,  that  collected  by  the  JSEP 
instructors,  and  information  gained  from  our  observations  and  interviews  of  JSEP 
participants  or  review  of  JSEP  materials. 
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The  modified  management  system  for  JSEP  at  the  PLATO  sites  automatically  collected 
background  information  about  soldiers  and  recorded  the  test  scores,  pass/fail  status,  and 
time  taken  in  each  lesson  that  a  participant  attempted.  Florida  State  University 
performed  this  modification  and  created  an  evaluation  data  set  to  which  AIR  had  access. 

Because  information  about  paper-based  lessons  was  not  collected  automatically  by  the 
JSEP  management  system,  instructors  maintained  logs  on  which  they  noted  paper  lessons 
attempted  and  test  scores  for  each  soldier.  The  logs  included  information  about  reasons 
for  exiting  the  program  (completed  prescription,  dropped  out,  and  so  forth)  and  notes  of 
any  supplementary  instruction  provided  to  the  soldier.  Data  from  the  logs  were  added  to 
the  lesson-level  data  sets. 

Instructors  pretested  and  posttested  soldiers  with  the  General  Academic  Predictor 
Test,  a  test  developed  by  the  Education  Center  staff  at  Fort  Lewis  to  parallel  the  tests 
comprising  the  GT  composite  of  the  Armed  Services  Vocational  Aptitude  Battery. 

Education  Center  staff  persons  from  the  JSEP  pilot  sites  had  used  this  test  for  several 
years  with  BSEP  and  vouched  for  its  validity  and  reliability.  This  test  served  as  the 
global  measure  of  verbal  and  mathematics  achievement  change  following  participation  in 
JSEP.  The  two  forms  of  the  test  were  counterbalanced  in  their  administration.  Four  sites 
pretested  soldiers  with  Form  A  and  posttested  them  with  Form  B,  while  the  schedule  was 
reversed  for  the  other  three  sites.  The  difficulty  of  the  two  forms  was  found  to  be  not 
equivalent  and  so  AIR  equated  the  scores  by  using  results  from  the  Locator  Test  that 
Florida  State  University  administered  as  a  pretest.  Equating  the  difficulties  prevented 
test  differences  from  artificially  inflating  or  deflating  gains.  Pretest  and  posttest  scores 
for  the  General  Academic  Predictor  (GAP)  Test  were  added  to  the  soldier  records  on  the 
data  set. 

Soldiers  responded  to  a  Post-JSEP  Attitude  Survey  when  they  completed  the  program. 
This  paper-and-pencil  instrument  included  questions  about  the  soldier’s  evaluation  of  JSEP, 
reasons  for  participating  in  it,  perceived  gains,  and  similar  topics.  Instructors 
administered  these  surveys  and  returned  them  to  AIR  for  analysis.  Finally,  instructors 
completed  a  short  paper-and-pencil  survey  giving  their  own  evaluation  of  JSEP  and 
recommendations  concerning  the  program.  (Copies  of  these  instruments  are  included  in 


Appendix  B.)  All  forms  and  tests  used  in  the  evaluation  were  provided  to  the  sites  by 
AIR  or  the  Army  Research  Institute. 

AIR  staff  visited  each  of  the  seven  evaluation  sites.  For  six  of  the  pilot 
installations,  a  team  of  two  evaluators  spent  two  or  two  and  one-half  days  observing  JSEP 
instruction  and  interviewing  a  range  of  participants.  We  typically  talked  with  soldiers 
enrolled  in  JSEP,  soldiers  who  had  completed  JSEP,  supervisors  of  both  kinds  of  soldier, 
and  instructors,  counselors,  and  administrators  in  the  local  Education  Centers.  The 
interviews  were  arranged  by  the  JSEP  Coordinator  at  each  site.  The  only  cases  in  which 
we  might  have  talked  with  a  nonrepresentative  sample  were  those  of  JSEP  graduates  or 
soldiers’  supervisors.  In  other  cases  we  interviewed  or  sampled  the  entire  existing 
population  (which  was  usually  no  larger  than  10  students  or  three  counselors).  From  our 
observations  the  only  bias  that  could  have  entered  into  the  selection  of  graduates  or 
supervisors  to  interview  was  that  of  availability.  We  asked  a  standard  set  of  core 
questions  at  each  site,  but  did  not  limit  interviews  to  these  predetermined  questions. 

Walter  Reed  Army  Medical  Center  was  not  treated  in  the  same  way  as  other  sites 
because  of  its  late  acquisition  of  JSEP.  Instead,  it  served  as  an  observational  "laboratory" 
for  the  AIR  evaluator  who  was  reviewing  JSEP  lessons.  Through  the  cooperation  of 
Walter  Reed  staff  and  soldiers  she  was  able  to  check  her  perceptions  of  the  lessons  with 
those  of  actual  students. 

The  Army  Research  Institute  and  Florida  State  University  provided  AIR  with  copies 
of  text  materials  to  review  and  with  access  to  JSEP  lessons  on  the  PLATO  system.  These 
are  discussed  in  detail  in  the  second  volume  of  this  report.  Part  II  Curriculum  Review- 

Scoring  and  equating  the  GAP  Test.  The  GAP  Test  was  originally  developed  to 
predict  BSEP  graduates’  performance  on  the  GT.  Calculating  scores  on  the  GAP  involved 
several  steps.  Raw  scores  on  the  verbal  and  arithmetic  sections  were  converted  to 
standard  scores;  these  were  then  summed  for  the  two  subject  areas;  and  the  resulting 
single  number  score  was  in  tum  converted  to  an  equivalent  on  the  GT. 

In  the  JSEP  evaluation,  soldiers  at  four  sites  wre  pretested  on  Form  A  of  the  GAP 
and  posttested  on  Form  B.  The  order  was  reversed  at  the  other  three  sites.  Each 
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student  was  also  pretested  with  a  single  form  of  the  Locator  Test,  which  measured  the 
same  areas  (verbal  and  arithmetic)  as  the  GAP. 

Correlating  the  GAP  and  Locator  Test  pretest  scores  revealed  that  the  two  forms  of 
the  GAP  were  not  equally  difficult.  The  discrepancy  was  limited  to  the  verbal  section  of 
the  GAP,  with  that  part  of  form  A  appearing  somewhat  more  difficult  than  Form  B. 

In  order  to  make  findings  from  the  GAP  comparable  across  the  evaluation  sites,  we 
had  to  adjust  the  GAP  scores  so  that  the  two  forms  were  of  equal  difficulty.  The 
procedure  was  basically  that  of  converting  scores  on  the  verbal  sections  of  Form  A  and  B 
to  the  same  standard.  (Conceptually,  this  was  a  bit  like  converting  Fahrenheit  to 
centigrade  so  that  readings  on  different  thermometers  would  agree  when  they  measured 
the  same  temperature.)  Using  the  standard  scores  (to  which  raw  scores  had  been 
converted)  we  calculated  the  pretest  mean  and  standard  deviation  on  the  verbal  section 
for  Form  A  (mean=45.47;  sd=5.55)  and  Form  B  (mean=45.82;  sd=4.95).  We  then  produced  a 
conversion  factor  using  the  formula: 

Mean  A  -  Mean  B  =  45.47  -  45.82  =  iUi  =  0.3 
SDa  /  SDb  5.55/4.95  1.12 

We  had  determined  that  Form  A  was  more  difficult  than  Form  B,  and  the  conversion 
factor  indicated  that  the  difference  was  0.3  standard  deviations.  In  other  words,  if  the 
tests  were  not  adjusted,  soldiers  who  took  Form  A  would  probably  achieve  a  score  1.5 
points  (0.3  x  4.95  =  1.485)  lower  than  if  they  had  taken  Form  B.  Therefore,  to  make  the 
two  tests  equivalent  in  difficulty,  we  added  1.5  to  the  standard  score  on  the  verbal 
section  for  all  Form  A  scores,  pretest  and  posttest. 

We  must  make  one  more  point  here.  This  adjustment  ensured  only  that  the  two 
forms  of  the  GAP  were  about  equal  in  difficulty.  Equating  did  not  affect  the  GAP’s 
relationship  to  the  GT  it  was  designed  to  mimic.  Anecdotal  reports  from  JSEP  instructors, 
and  informal  review  of  GT  scores  from  soldiers  tested  after  completing  JSEP,  suggest  that 
the  GAP  is  more  difficult  than  the  GT.  The  correlation  between  the  entering  GT  scores 
and  pre-JSEP  GAP  Test  scores  for  the  soldiers  in  this  evaluation  was  0.24;  statistically 
unlikely  to  have  occurred  by  chance,  but  still  a  relatively  weak  relationship.  Whether  this 
relationship  reflects  actual  differences  between  the  two  tests  or  incidental  learning  that 
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had  taken  place  between  the  times  each  was  administered  cannot  be  determined.  The 
correlation  between  the  post-JSEP  GAP  and  GT  for  61  soldiers  was  a  slightly  higher  0.33. 

Data  management  and  analysis.  AIR  reviewed  data  collected  automatically  on  JSEP 
for  any  anomalies  or  unexpected  values,  and  referred  questions  about  these  to  Florida 
State  University  for  resolution.  Data  processed  by  AIR,  such  as  GAP  Test  scores,  were 
verified  and  examined  for  unexpected  outlier  values.  Both  members  of  each  site  visit  team 
wrote  their  field  notes  independently  and  then  reviewed  the  pooled  notes  to  identify  any 
areas  of  confusion  or  disagreement. 

Quantitative  data  were  analyzed  using  the  Statistical  Analysis  System  (SAS)  or  the 
Statistical  Package  for  the  Social  Sciences  (SPSS).  The  specific  procedures  and  tests  used 
are  noted  for  each  set  of  findings. 

The  chapters  that  follow  present  the  findings  collected  through  these  evaluation 
activities  and  address  the  evaluation  questions.  Chapter  3  presents  the  quantitative 
analyses  of  soldier  performance  data.  Chapter  4  combines  the  site  visit  and  interview 
data  with  the  results  of  attitudinal  surveys  to  provide  an  explanatory  context  for  the 
performance  results. 
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Chapter  3.  JSEP  Learning  Outcomes 


The  background  characteristics,  overall  reading  and  mathematics  achievement,  and 
lesson-level  performance  of  soldiers  in  the  JSEP  evaluation  are  reported  in  this  chapter. 

The  discussion  begins  by  describing  the  soldiers  in  the  evaluation  so  that  the  reader  may 
judge  how  generalizable  the  findings  are. 

Soldiers  Participating  in  Evaluation 

As  we  described  in  Chapter  2,  during  the  evaluation  th.e  automated  soldier 
management  system  was  altered  by  Florida  State  University  to  collect  lesson-level  pretest 
and  other  data  from  soldiers  using  JSEP  at  PLATO  sites.  We  were  able  to  retrieve 
information  on  179  soldiers  from  four  of  these  five  sites.  (Unfortunately,  the  data  from 
Walter  Reed  Army  Medical  Center  could  not  be  merged  with  those  from  other  sites.)  An 
additional  group  of  44  soldiers  provided  more  limited  data  from  the  two  MicroTICCIT  sites. 
The  performance  data  presented  in  this  chapter  are  drawn  from  the  four  PLATO  sites 
from  which  lesson-level  information  was  available,  unless  otherwise  noted4. 

The  background  characteristics  of  the  179  soldiers  enrolled  in  JSEP  during  the 
evaluation  are  shown  on  Table  3.1.  The  largest  number  (88)  came  from  Fort  Sill,  and 
about  85  percent  were  male.  Slightly  over  half  (53%)  were  black;  somewhat  fewer  than 
one  in  five  (18%)  reported  earlier  experience  in  BSEP. 

The  largest  number  of  soldiers  (88%)  reported  having  completed  exactly  12  years  of 
formal  education.  Only  2  percent  had  less  schooling  than  this.  The  group  was  generally 
young,  with  four  out  of  five  (80%)  in  the  age  20  to  29  category.  In  accord  with  this  age 
distribution,  the  most  commonly  reported  rank  was  E4  (44%),  followed  by  E5  (22%)  and  E3 


Although  learning  outcome  and  lesson-level  data  from  MicroTICCIT  participants 
were  not  used  in  this  report,  they  were  forwarded  informally  to  the  Army  Research  Institute. 
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Table  3.1.  Background  Characteristics  of 
JSEP  Evaluation  Participants  (N-179) 


Characteri stic 

Number 

Percent 

Site 

Fort  Sill 

88 

49.2 

Fort  B1 iss 

37 

20.7 

Fort  Jackson 

34 

19.0 

Fort  Leonard  Wood 

20 

11.2 

Gender 

Male 

151 

84.4 

Female 

28 

15.6 

Ethnic  Background 

Black 

94 

53.4 

White 

61 

34.7 

Other 

21 

11.9 

Missing  data 

3 

-  - 

Years  of  Education 

Fewer  than  12  years 

3 

1.9 

12  years 

143 

88.3 

13  to  15  years 

15 

9.3 

16  years 

1 

0.6 

Missing  data 

17 

-- 

Prior  BSEP  Enrollment 

Yes 

33 

18.4 

No 

146 

81.6 

Age 

19  or  younger 

12 

6.7 

20  to  29 

142 

79.8 

30  to  39 

24 

13.5 

Missing  data 

1 

Rank 

El 

1 

0.6 

E2 

13 

7.4 

E3 

35 

19.9 

E4 

78 

44.3 

E5 

38 

21.6 

E6 

11 

6.3 

Missing  data 

3 

•  - 
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Table  3.1.  (Cont.)  Background  Characteristics 
of  JSEP  Evaluation  Participants  (N=l 79) 


Characteri Stic 

Number 

Percent 

Years  in  Service  (Mean=3.9) 

Less  than  1 

23 

13.2 

1 

43 

24.7 

2 

30 

17.2 

3 

19 

10.9 

4 

10 

5.7 

5 

13 

7.5 

6 

5 

2.9 

7 

4 

2.3 

8 

7 

4.0 

10  or  more 

20 

11.6 

Years  to  Separation  (Mean-1.8) 

Less  than  1 

51 

30.4 

1 

58 

34.5 

2 

30 

17.9 

3 

14 

8.3 

4 

8 

4.8 

5 

7 

4.2 

Missing  data 

11 

-- 
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(20%).  The  soldiers  reported  an  average  of  3.9  years  experience  in  the  Armed  Services, 
with  about  half  (55%)  having  served  two  or  fewer  years.  On  the  average  they  reported 
1.8  years  to  separation,  with  two  out  of  three  (65%)  listing  expected  separation  dates  less 
than  two  years  away. 

Table  3.2  lists  the  background  achievement  levels  of  the  participating  soldiers.  The 
average  p  re- JSEP  GT  score  for  the  group  was  92.8,  with  about  one  in  five  (19%)  soldiers 
meeting  the  criterion  of  100  generally  required  for  reenlistment  eligibility.  Scores  on  the 
Tests  of  Adult  Basic  Education  (TABE)  were  available  for  three-fourths  of  the  group 
(75%).  Among  these  soldiers,  the  average  reading  score  was  at  the  10.0  grade  level  and 
the  average  mathematics  score  was  at  the  8.7  grade  level.  As  a  benchmark  of  what  these 
scores  mean,  Army  Regulation  621-5  sets  as  its  objective  for  BSEP  II  the  achievement  of 
scale  scores  equivalent  to  the  10.1  grade  level  in  reading  and  math  by  20  percent  or  more 
of  the  graduates.  TABE  language  scores,  reported  for  a  smaller  number  of  participants 
(61%),  averaged  8.7  grade  level. 

In  summary,  the  educational  background  of  these  soldiers  was  higher  than  had  been 
expected  -  almost  all  had  completed  high  school  and  most  read  at  a  tenth  grade  level  or 
better  -  but  was  generally  within  the  parameters  established  for  JSEP  eligibility.  Those 
criteria,  stated  in  an  August  1987  memorandum  from  the  Soldier  Education  Division,  said 
that  soldiers  would  be  eligible  for  instruction  if  they  did  not  attain  the  cut  score  on  the 
JSEP  pretest  (no  such  test  was  in  effect  at  the  time  of  the  evaluation)  or  did  not  meet 
reenlistment  requirements.  Since  a  GT  score  of  100  or  better  is  a  common  reenlistment 
requirement,  81  percent  of  the  group  was  eligible  for  JSEP  on  the  basis  of  this  criterion. 

The  other  19  percent  may  well  have  met  an  additional  reenlistment  criterion  that  was  not 
recorded. 

Measuring  Soldier  Performance 

Two  levels  of  information  about  academic  performance  were  collected  for  the  soldiers 
participating  in  the  evaluation.  One  consisted  of  the  lesson-level  data  recorded 
automatically  by  JSEP’s  soldier  management  system.  The  other,  discussed  in  the  following 
paragraphs,  included  more  global  information  about  reading  and  mathematics  achievement 
collected  by  the  GAP  Test,  which  was  designed  to  parallel  the  GT. 
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Table  3.2.  Entry  Academic  Achievement 
Levels  of  Soldiers  (N— 179) 


Test/Area 

Number 

Percent 

Mean 

SD 

ASVAB/GT  Range 

92.8 

12.4 

0--49 

2 

1.2 

50--69 

-- 

-  - 

70--89 

55 

32.2 

90--99 

82 

48.0 

100  —  1 09 

28 

16.4 

110  or  above 

4 

2.3 

Missing  data 

8 

-- 

TABE/Reading  Range 

10.0 

1.5 

Grade  5.0--6.9 

4 

3.0 

Grade  7.0- -8. 9 

31 

23.0 

Grade  9.0--10.9 

70 

51.9 

Grade  11.0--12.9 

30 

22.2 

Missing  data 

44 

-- 

TABE/Math  Range 

8.7 

1.6 

Below  grade  5.0 

1 

0.7 

Grade  5.0--6.9 

12 

8.9 

Grade  7.0--8.9 

70 

51.9 

Grade  9.0--10.9 

38 

28.1 

Grade  11.0--12.9 

14 

10.4 

Missing  data 

44 

-- 

TABE/language  Range 

8.7 

1.8 

Grade  5.0--6.9 

17 

15.6 

Grade  7.0--8.9 

50 

45.9 

Grade  9.0--10.9 

31 

28.4 

Grade  11.0--12.9 

11 

10.1 

Missing  data 

70 

-  - 
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Pre-  and  post-JSEP  tests.  The  JSEP  instructors  administered  the  GAP  Test  to 
evaluation  participants  immediately  before  they  began  and  after  they  completed  their  J SEP 
prescriptions.  Chapter  2  discusses  this  test  and  notes  that  there  was  a  difference  in  the 
difficulty  of  the  two  forms  that  were  used.  The  information  reported  in  this  chapter  is 
based  upon  GAP  scores  adjusted  to  equate  the  two  forms;  this  was  done  so  that  changes 
measured  by  the  tests  would  not  be  inaccurately  inflated  or  depressed. 


Performance  Change:  GAP  Results 

The  results  for  the  GAP  are  shown  on  Table  33.  Although  pretests  were  reported 
for  171  of  the  soldiers,  the  evaluation  ended  before  they  had  all  completed  their 
prescriptions,  and  posttest  results  were  thus  available  for  only  133.  Changes  were 
calculated  by  subtracting  an  individual’s  pretest  score  from  his  or  her  posttest  score, 
excluding  soldiers  for  whom  only  one  test  score  was  given. 


Table  3.3.  Pre-  and  Post-JSEP  Performance  on 
General  Academic  Predictor  Test  (N-179) 


Pre-JSEP 

Post-JSEP 

Change 

!* 

Scores 

Mean  SD 

.  .  Mean  SD 

Mean 

SD 

Language  standard  score 

46.0  5.4 

46.6  5.5 

0.5 

4.7 

Mathematics  standard  score 

46.9  5.5 

49.6  6.3 

2.4 

5.4 

Total -GT  conversion 

92.3  9.7 

95.9  9.9 

3.1 

8.4 

Missing-number 

(8)  - 

(46)  - 

Pre-JSEP 

Post-JSEP 

GT  Conversion  Ranae 

No.  Pet. 

No.  Pet. 

50--69 

3  1.8 

70--89 

67  39.2 

32  24.1 

90--99 

67  39.2 

56  42.1 

100 — 109 

30  17.5 

30  22.6 

110  or  above 

4  2.3 

15  11.3 

Missing-number 

(8) 

(46) 

♦Calculated  for  133  soldiers  for  whom  pretest  and  posttest  were 
available. 
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An  examination  of  the  standard  scores  shows  that  gains  were  greater  in  mathematics 
(2.4  standard  score  points)  than  in  reading  (0.5  points).  The  standard  deviation  is  the 
range  above  and  below  the  mean  into  which  about  two-thirds  of  the  cases  fall.  The 
figure  for  mathematics,  for  example,  (SD=5.4)  means  that  about  two-thirds  of  the  soldiers 
had  changes  ranging  from  a  loss  of  three  points  to  a  gain  of  eight  points. 

When  the  standard  scores  in  reading  and  mathematics  were  summed  and  converted  to 
a  GT  score,  the  overall  average  gain  was  about  3  points.  The  soldiers  entered  JSEP  with 
an  average  GAP  score  of  923.  They  completed  with  an  average  score  of  95.9.  Matching 
the  pre-  and  posttest  scores  of  those  for  whom  both  were  available  produced  a  change  of 
3.1  points. 

The  soldiers  for  whom  posttest  scores  were  available  had  slightly  higher  pretest 
scores  than  did  soldiers  who  were  not  posttested:  an  average  of  92.8  as  opposed  to  90.6. 

The  second  half  of  the  table  looks  at  the  GAP  score  ranges  in  the  pretest  and  posttest 
groups.  About  one-fifth  of  the  pretest  group  (20%)  had  scores  of  100  or  better  on  the 
GAP.  This  proportion  was  increased  to  one-third  (34%)  among  the  posttest  group. 

Average  gains  are  not  the  same  as  the  number  of  persons  meeting  a  criterion.  A  small 
gain  may  be  enough  to  push  sizable  numbers  over  a  cut-score,  and  this  appears  to  be 
what  happened  with  the  GAP  results. 

Factors  Associated  with  GAP  Results 

We  next  looked  at  factors  that  seemed  possibly  related  to  soldiers’  performance  on 
the  GAP,  using  analysis  of  variance.  This  is  a  statistical  technique  that  addresses  the 
question:  how  much  of  the  variance  in  an  outcome  measure  (here,  the  differences  among 
soldiers’  GAP  posttest  scores)  can  we  explain  by  looking  at  the  values  of  plausibly  related 
factors? 

Background  factors.  Table  3.4  reports  an  analysis  of  variance  test  of  the 
relationship  between  GAP  posttest  and  several  background  factors:  the  site  at  which  the 
program  was  held,  and  the  soldier’s  rank,  years  of  service,  formal  education,  and  GAP 
pretest  score. 
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Table  3.4.  Variance  in  Soldiers'  General  Academic  Predictor  (GAP) 
Posttest  Associated  with  Background  Factors  (N-112) 


Source 

Degrees 

Freedom 

Sum  of 
Sauares 

Mean 

Sauare 

F  Value 

PR>F 

R-Sauare 

Model 

16 

5204.4 

325.3 

5.21 

0.0001 

0.467186 

Error 

95 

5935.5 

62.5 

Corrected  total 

111 

11139.9 

Site 

3 

466.2 

2.49 

0.0652 

Rank 

5 

322.1 

1.03 

0.4040 

Years  in  service 

1 

96.3 

1.54 

0.2174 

Years  education 

6 

268.8 

0.72 

0.6368 

GAP  pretest 

1 

2981.2 

47.72 

0.0001 

Gtoud 

Number 

Avg.  GAP 
oretest 

Avg.  GAP 
Dosttest 

Avg. 

chanoe 

Rank 

El 

1 

97.0 

99.0 

2.0 

E2 

6 

93.7 

91.7 

-2.0 

E3 

26 

92.7 

95.2 

2.5 

E4 

50 

94.5 

97.5 

3.0 

E5 

21 

90.3 

95.0 

4.7 

E6 

8 

95.3 

96.1 

0.8 

Years 

9 

Education 

1 

95.0 

105.0 

10.0 

11 

1 

84.0 

85.0 

1.0 

12 

99 

93.3 

95.8 

2.5 

13 

6 

91.8 

98.3 

6.5 

14 

2 

103.0 

106.0 

3.0 

15 

2 

86.0 

92.5 

6.5 

16 

1 

109.0 

105.0 

-4.0 
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The  combination  accounted  for  about  47  percent  of  the  variation  in  GAP  posttest 
scores  (R-square=  0.467)  and  the  probability  that  this  relationship  would  have  occurred  by 
chance  was  less  than  0.0001.  However,  when  the  factors  were  examined  individually,  only 
one  met  the  <.05  probability  (5  or  fewer  chances  in  100)  established  as  the  significance 
level  for  this  study.  That  factor  was  GAP  pretest,  and  the  relationship  was  positive:  the 
higher  a  soldier’s  pretest  score  on  the  GAP,  the  higher  the  posttest  score.  The  bottom 
half  of  Table  3.4  shows  the  average  pre-  and  posttest  scores  and  gains  for  soldiers  at 
different  ranks  and  levels  of  education.  This  illustrates  that  there  was  no  consistent 
relationship  between  these  factors  and  performance  on  the  GAP. 

Program  factors.  Table  3 3  carries  out  the  same  statistical  analysis  with  two 
programmatic  factors:  the  time  spent  in  JSEP  and  the  number  of  lessons  attempted.  This 
table  reports  analyses  of  variance  for  the  GAP  posttest  language  and  mathematics  standard 
scores  as  well  as  for  the  GT  conversion  score. 

The  number  of  lessons  taken,  time  in  JSEP,  and  pretest  language  score  explain  about 
40  percent  (R-square=  0.404)  of  the  variance  in  language  posttest  scores,  at  a  probability 
level  of  less  than  0.0001.  However,  the  only  statistically  significant  factor  in  the  model  is 
language  pretest  score. 

For  mathematics,  the  total  model  explains  42  percent  of  the  variance  in  posttest 
scores  and  each  of  the  factors  meets  the  .05  significance  level.  Time  in  JSEP,  number  of 
lessons  attempted,  and  mathematics  pretest  score  are  associated  with  posttest  performance 
to  a  degree  that  is  not  very  likely  due  to  chance.  The  relationship  between  time  and 
posttest  score  is  negative:  the  less  time  spent  in  JSEP,  the  higher  the  mathematics 
posttest  score. 

The  outcomes  were  the  same  in  analyzing  overall  GT  conversion  scores.  The 
combination  of  factors  examined  explained  46  percent  of  the  variance  at  a  probability 
level  of  less  than  0.0001.  The  relationship  was  statistically  significant  for  time  in  JSEP 
(0.0011)  and  GAP  pretest  score  (0.0001).  The  relationship  between  posttest  score  and  time 
in  JSEP  was  also  negative. 

In  summary,  soldiers  showed  gains  of  about  3  points  on  the  GAP  following 
participation  in  JSEP.  The  program  had  a  greater  effect  on  mathematics  performance  than 
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Table  3.5.  Variance  in  Soldiers'  General  Academic  Predictor  (GAP) 
Posttest-Language  Score  Associated  with  Time  in  and 
Number  of  Lessons  ( N= 1 29 ) 


Source 

Degrees 

Freedom 

Sum  of 
Sauares 

Mean 

Sauare 

F  Value 

PR>F 

R-Sauare 

Lanauaae  Score 

Model 

3 

1591.5 

530.5 

28.19 

0.0001 

0.403545 

Error 

125 

2352.4 

18.8 

Corrected  total 

128 

3944.0 

Time  in  JSEP* 

1 

25.6 

1.36 

0.2459 

Number  lessons* 

1 

4.3 

0.23 

0.6324 

GAP  language  pretest 

1 

1251.7 

66.51 

0.0001 

Mathematics  Score 

Model 

3 

2166.9 

722.3 

30.70 

0.0001 

0.424257 

Error 

125 

2940.6 

23.5 

Corrected  total 

128 

5107.6 

Time  in  JSEP* 

1 

311.1 

13.22 

0.0004 

Number  lessons 

1 

109.7 

4.66 

0.0327 

GAP  math  pretest 

1 

681.8 

28.98 

0.0001 

GT  Conversion  Score 

Model 

3 

5794.5 

1931.5 

35.26 

0.0001 

0.458368 

Error 

125 

6847.1 

54.8 

Corrected  total 

128 

12641.5 

Time  in  JSEP* 

1 

611.2 

11.16 

0.0011 

Number  lessons 

1 

114.0 

2.08 

0.1516 

GAP  GT  pretest 

1 

2231.8 

40.74 

0.0001 

♦Negative  relationship; 

e.g.,  the  longer  the 

time  in 

JSEP,  the  lower 

the 

posttest  score. 
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on  reading.  Two  factors  appeared  to  influence  posttest  results.  These  were  pretest 
scores  and  the  length  of  time  spent  in  JSEP,  with  soldiers  who  spent  less  time  in  the 
program  achieving  higher  posttest  scores.  This  last  could  have  reflected  some  general 
ability,  such  as  learning  speed.  The  question  of  whether  greater  GAP  gains  could,  or 
should,  have  been  expected  is  discussed  in  the  closing  chapter  on  results  and  conclusions. 


Lesson  Level  Results 

The  lesson-level  information  came  from  the  demographics  and  lesson  performance 
results  recorded  by  JSEP’s  soldier  management  system,  and  from  records  of  paper  lesson 
performance  maintained  by  the  JSEP  instructors. 

Match  Between  Prescriptions  and  MOS  s 

The  JSEP  prescriptions  are  intended  to  cover  the  most  heavily  populated  MOSs. 
JSEP  assigns  a  prescription  for  the  soldier’s  primary  MOS;  if  none  exists,  for  his  or  her 
secondary  MOS;  and  if  none  is  available  for  that  MOS,  the  program  prescribes  the 
Common  Core  lessons.  Table  3.6  lists  the  number  of  soldiers  in  each  primary  and 
secondary  MOS  reported  among  the  group,  and  indicates  whether  there  was  a 
corresponding  JSEP  prescription.  There  were  prescriptions  for  the  primary  MOS  of  125 
(70%)  of  the  soldiers,  and  for  the  secondary  MOS  of  23  (68%)  of  those  soldiers  who 
reported  one.  The  JSEP  participants  reflected  a  wide  range  of  MOSs,  with  about  70 
percent  from  the  Combat  Support  or  Combat  Service  Support  Career  Management  Fields. 
About  half  of  those  remaining  were  drawn  from  the  Artillery  Career  Management  Field. 
Among  those  without  a  prescription,  the  MOSs  with  the  greatest  numbers  of  soldiers  were 
76 Y,  Unit  Supply  Specialist  (18  persons)  and  3  IK,  Combat  Signaller  (10  persons). 

Learning  Strategies  Lessons 

The  Learning  Strategy  Lessons  do  not  teach  specific  MOS-related  content,  but  cover 
general  learning  skills  that  are  believed  to  contribute  to  a  soldier’s  success  in  completing 
JSEP.  Adi  soldiers  are  automatically  assigned  the  Time  Management  lesson  when  they 
begin  JSEP.  The  other  lessons  may  be  assigned  by  the  instructor  or  automatically 
presented  to  students  when  their  performance  on  content  lesson  tests  trips  a 
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Table  3.6.  Match  Between  JSEP  Prescriptions  and  Soldiers' 

Military  Occupation  Specialties  (N*179) 

MOS  Title 

Number  and 
Primary  MOS 
No.  Pet. 

Percent  Reporting  as: 
Secondary  MOS 

No.  Pet. 

JSEP 

Prescription? 

lib  Infantryman 

7 

3.9 

8 

4.5 

Yes 

11c  Indirct  fire  infantry 

-- 

-- 

1 

0.6 

Yes 

12b  Combat  engineer 

13 

7.3 

2 

1.1 

Yes 

13b  Cannon  crewmember 

23 

12.8 

1 

0.6 

Yes  ! 

13e  Cannon  fire  spec 

1 

0.6 

Yes 

13f  Fire  support  spec 

1 

0.6 

Yes 

13m  Multi  launch  rocket 

2 

1.1 

No 

15e  PERSHING  crewmember 

1 

0.6 

Yes 

16d  HAWK  crewmember 

4 

2.2 

Yes 

16p  CHAPARREL  crewman 

-- 

-- 

1 

0.6 

Yes 

16r  CHAPARREL  crewmember 

3 

1.7 

No 

16s  MANPADS  crewmember 

1 

0.6 

No 

16t  PATRIOT  crewmember 

1 

0.6 

No 

19d  Cavalry  scout 

1 

0.6 

1 

0.6 

Yes 

24m  VULCAN  mech 

1 

0.6 

No 

24t  PATRIOT  operator/mech 

1 

0.6 

No 

29n  Phone  cntrl  off  repair 

1 

0.6 

No 

31c  Single  chan  radio  op 

2 

1.1 

No 

31k  Combat  signaller 

10 

5.6 

No 

311  Wire  system  install 

1 

0.6 

Yes 

31m  Multichannel  comm 

1 

0.6 

1 

0.6 

Yes 

31v  Tactic  commn  op/mech 

-- 

-- 

2 

1.1 

Yes 

451  Artillery  repair 

1 

0.6 

No 

45t  BRADLEY  turret  mech 

1 

0.6 

No 

51b  Carpentry/mason  spec 

1 

0.6 

No 

51m  Firefighter 

1 

0.6 

No 

52d  Power  gen  equip  repair 

3 

1.7 

Yes 

54b  Chemical  oper  spec 

2 

1.1 

No 

55b  Ammunition  spec 

-- 

-- 

1 

0.6 

Yes 

57h  Cargo  spec 

-- 

-- 

2 

1.1 

No 

62b  Constrct  equip  repair 

2 

1.1 

Yes 

62e  Hvy  constrct  oper 

2 

1.1 

Yes 

62h  Concrt/asphalt  equip 

1 

0.6 

No 

62n  Constrct  equip  super 

1 

0.6 

No 

63b  Light  vehicle  mech 

9 

5.0 

3 

1.7 

Yes 

63d  Sp  FA  sys  mech 

2 

1.1 

No 

63e  Ml  ABRAMS  mech 

1 

0.6 

No 

63s  Hvy  vehicle  mech 

3 

1.7 

No 

63w  Water  treat  spec 

3 

1.7 

Yes 

64c  Motor  transport  oper 

7 

3.9 

3 

1.7 

Yes 

67v  Obs/scout  heli  repair 

1 

0.6 

Yes 

711  Admin  spec 

12 

6.7 

2 

1.1 

Yes 

71m  Chapel  activs  spec 

1 

0.6 

No 

3 


Table  3.6.  (Cont.)  Match  Between  JSEP  Prescriptions  and  Soldiers' 
Mil itary  Occupation  Specialties  (N=179) 


Number  and  Percent  Reporting  as:  JSEP 

Primary  MOS  Secondary  MOS  Prescription? 


MOS  Title 

No . 

Pet. 

No. 

Pet. 

72e  Telecomm  center  op 

1 

0.6 

No 

75b  Personnel  admin  spec 

3 

1.7 

Yes 

75c  Personnel  mgt  spec 

3 

1.7 

No 

76c  Equip  records/parts 

4 

2.2 

Yes 

76p  Material  control/acct 

2 

1.1 

Yes 

76v  Material  store/handle 

1 

0.6 

Yes 

76x  Subsist  supply  spec 

-- 

1 

0.6 

Yes 

76y  Unit  supply  spec 

17 

9.5 

1 

0.6 

No 

77 f  Petrol  supply  spec 

1 

0.6 

1 

0.6 

No 

88n  Traffic  mgt  coord 

1 

0.6 

No 

91a  Medical  spec 

2 

1.1 

1 

0.6 

No 

91b  Medical  NCO 

1 

0.6 

Yes 

91c  Practical  nurse 

1 

0.6 

No 

91d  Operate  room  spec 

1 

0.6 

No 

94b  Food  service  spec 

9 

5.0 

Yes 

95b  Military  police 

1 

0.6 

1 

0.6 

Yes 

95c  Correctional  spec 

1 

0.6 

No 

No  MOS  reported 

-- 

-- 

145 

81.0 

default.  For  example,  the  Problem  Solving  lesson  is  presented  to  a  student  after  she  or 
he  has  failed  a  preestablished  number  of  mathematics  lessons  (those  in  Series  1  through 
19). 


The  number  of  soldiers  assigned  these  lessons,  and  the  time  to  complete  them,  are 
shown  on  Table  3.7.  All  soldiers  were  assigned  to  Time  Management,  and  82  percent  of 
them  to  Test-Taking.  While  over  half  (52%)  were  given  the  Problem  Solving  lesson,  only  a 
few  (4%)  took  Reading  Strategies.  None  of  the  soldiers  was  assigned  the  lesson  on 
Motivational  Skills.  On  the  average,  it  required  77  minutes  to  complete  Time  Management; 
the  fastest  soldier  completed  the  lesson  in  a  little  over  one-half  hour  (32  minutes),  while 
the  slowest  took  about  three  hours  (189  minutes)  to  finish  it.  Problem  Solving  required 
an  average  of  99  minutes;  Test  Taking  an  average  of  71.  The  ranges  between  minimum 
and  maximum  times  are  typical  of  those  observed  for  many  lessons. 


Table  3.7.  Learning  Strategies;  Number  Attempting 
Lessons  and  Time  to  Complete  (N-179) 


Lesson 

Assigned 

No.  Pet. 

Time 

Ava. 

in  Minutes 
Min.  Max. 

LSI. 

Time  Management 

179 

100.0 

77 

32 

189 

LS2. 

Problem  Solving 

93 

52.0 

99 

12 

322 

LS3. 

Reading  Strategies 

7 

3.9 

52 

29 

81 

LS4. 

Test  Taking 

146 

81.6 

71 

31 

260 

LS5. 

Motivational  Skills 

0 

“  “ 

”  - 

-  - 
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Lesson  Test  Results 

Florida  State  University  modified  the  JSEP  soldier  management  system  for  the 
evaluation  so  that  soldiers  were  pretested  before  they  began  each  on-line  lesson.  Every 
prerequisite  competency  had  its  own  brief  test;  but  for  any  given  competency  the  same 
items  comprised  the  pretest  and  the  posttests  for  both  the  short  and  long  lessons.  The 
possible  combinations  of  testing  and  instruction  were  the  following. 


Pretest 

Pretest 

Short  Lesson 

Short  Lesson 

Lesson 

Posttest 

Posttest 

Posttest 

Long  Lesson 

Posttest 

It  should  be  noted  that  changing  the  soldier  management  system  to  include  pretests  meant 
that  soldiers  were  allowed  only  one  attempt  at  the  long  lesson  instead  of  the  two 
attempts  that  are  part  of  JSEP’s  design.  As  a  result,  the  proportions  of  soldiers  failing 
long  lessons  in  this  study  are  possibly  larger  than  they  would  be  if  soldiers  were  given  a 
second  chance  at  the  lesson. 

The  system  recorded  each  soldier’s  status  for  every  on-line  lesson  he  or  she 
attempted.  The  status  recorded  for  a  lesson  could  be  that  the  soldier  passed  the  pretest, 
passed  the  short  lesson  test,  passed  the  long  lesson  test,  or  failed.  There  were  no 
pretests  or  enforced  passing  scores  for  paper  lessons,  and  the  system  recorded  only  if 
these  had  been  attempted.  We  added  the  paper  lesson  scores  reported  by  the  instructors 
to  the  lesson  files  for  the  evaluation. 

Summary  lesson  data.  The  summary  statistics  for  the  lesson-level  performance  and 
time  are  shown  on  Table  3.8;  subsequent  tables  included  in  Appendix  C  break  this 
information  out  in  more  detail.  Table  3.8  lists  each  JSEP  lesson,  indicating  whether  it 
was  on-line  or  paper.  The  table  shows  the  number  of  soldiers  attempting  each  lesson, 


5For  brevity,  Diagnostic  Review  Lessons  will  be  referred  to  as  "short"  and  Skill 
Development  Lessons  as  "long"  lessons. 
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Table  3.8.  Number  and  Percent  of  Soldiers  Attempting  Each  Lesson 
with  Summary  Performance  Statistics  (N-179) 


Lesson 

Attempt 

Lesson 

(No.) 

Passed 

Pretest 

(Pet.) 

Passed 

Short* 

(Pet.) 

Passed 

Long 

(Pet.) 

Minutes  to 
Failed**  Complete 
(Pet.)  Min  Max 

Numbering  &  Counting 
la.  Match  name 

175 

69.7 

20.0 

8.0 

2.3 

8 

221 

lb. 

Sequence 

174 

35.1 

23.6 

24.1 

17.2 

8 

225 

1c. 

Before,  between 

175 

3.4 

22.9 

26.3 

47.4 

10 

594 

Id. 

Greater/lesser 

174 

47.7 

17.8 

19.5 

14.9 

3 

526 

le. 

Ordinal  position 

174 

77.6 

18.4 

2.9 

1.1 

5 

163 

If. 

Place  value 

175 

49.1 

46.9 

1.7 

2.3 

4 

92 

lg. 

Round  whole/decimal 

174 

27.0 

42.0 

19.5 

11.5 

5 

293 

lh. 

Count  by  1,5,10,  etc. 

171 

29.2 

45.6 

-- 

25.1 

6 

148 

li. 

Match  scale  intervals 

172 

29.1 

25.6 

-- 

45.3 

7 

391 

Linear,  Weight  &  Volume 
Measures 

2a.  Linear  scale  mark 

167 

84.4 

9.0 

6.0 

0.6 

4 

85 

2b. 

US  &  metric  measures 

169 

48.5 

45.6 

5.9 

0.0 

3 

442 

2c. 

Lengths 

167 

85.6 

13.2 

1.2 

0.0 

5 

133 

2d. 

Weight,  pressure,  torque 

53 

88.7 

9.4 

-- 

1.9 

3 

41 

2e. 

Volume 

169 

60.4 

31.4 

-- 

8.3 

6 

165 

2f . 

Non-numeric  calibrated 

167 

61.1 

35.9 

-- 

3.0 

3 

91 

2g. 

Estimate  size,  distance 

167 

21.0 

42.5 

10.2 

26.3 

5 

239 

Degree  Measures 
3a.  Degrees,  mils,  angles, 


temp 

164 

64.0 

30.5 

4.9 

0.6 

6 

202 

3b.  Est  angle  <180  deg 

164 

62.8 

25.0 

6.7 

5.5 

4 

205 

3c.  Interp  azimuths,  0-6400 
mils 

162 

5.6 

35.8 

35.8 

22.8 

5 

420 

3d.  Interp  azimuths,  0-360 
deg 

162 

53.1 

34.6 

7.4 

4.9 

3 

176 

Time-Telling  Measure? 

4a.  Digital,  analog,  24  hr 

169 

11.2 

60.9 

21.9 

5.9 

4 

318 

4b.  Clockface  direction 

168 

50.0 

45.8 

4.2 

0.0 

3 

66 

4c.  Est  secs,  mins 

168 

81.0 

14.3 

1.8 

3.0 

4 

107 

4d.  Gregorian  &  Julian 

170 

1.2 

18.2 

48.2 

32.4 

19 

442 

4e.  Convert  hours,  lOths 

45 

0.0 

71.1 

-- 

28.9 

18 

128 

4f.  Convert  to  Zulu 

3 

0.0 

0.0 

33.3 

66.7 

140 

228 

*Paper  lessons  indicated  by  asterisk;  score  shown  as  100.0  in  "Passed  Short" 
column. 

**Percent  failing  test  after  final  lesson  for  prerequisite  competency;  dashes  in 
"Passed  Long"  column  indicate  no  long  lesson  for  competency. 
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Table  3.8.  (Cont.)  Number  and  Percent  of  Soldiers  Attempting  Each  Lesson 
with  Summary  Performance  Statistics  (N-179) 


Lesson 

Attempt 

Lesson 

(No.) 

Passed 

Pretest 

(Pet.) 

Passed 

Short* 

(Pet.) 

Passed 

Long 

(Pet.) 

Minutes  to 
Failed**  Complete 
(Pet.)  Min  Max 

Gaoe 

5a. 

Measures 

Read  gage 

162 

17.3 

69.8 

11.1 

1.9 

7 

233 

5b. 

Read  read-out 

162 

70.4 

27.8 

1.2 

0.6 

3 

70 

5c. 

Read  gage,  color 

162 

81.5 

17.9 

0.6 

0.0 

4 

32 

5d. 

Read  scales  (+/•) 

162 

97.5 

1.9 

0.6 

0.0 

3 

51 

5e. 

Read  multi  scale 

162 

63.6 

18.5 

16.0 

1.9 

5 

205 

5f . 

Match  gage  to  spec 

161 

75.2 

16.8 

4.3 

3.7 

4 

108 

5g. 

Read  unnumbered/unmarked  161 

96.3 

2.5 

1.2 

0.0 

3 

32 

5h. 

Read  fluctuating  gage 

161 

52.8 

35.4 

6.2 

5.6 

9 

188 

5i . 

Match  specs/aline 

161 

90.7 

9.3 

0.0 

0.0 

5 

43 

SDatial 

6a.  Ident  direct  tools  move 

155 

98.7 

1.3 

0.0 

0.0 

5 

61 

6b. 

Manip  to  aline,  etc. 

154 

33.8 

40.3 

14.9 

11.0 

10 

216 

6c. 

2-dimension  to  spatial 

154 

95.5 

3.2 

1.3 

0.0 

6 

242 

6d. 

Symbols  to  systems 

154 

47.4 

31.8 

-- 

20.8 

6 

64 

Lines 

7a. 

Ident  points,  etc. 

157 

43.9 

52.9 

3.2 

0.0 

3 

46 

7b. 

Ident  vert,  horiz,  diag 

157 

58.0 

34.4 

5.7 

1.9 

3 

197 

7c. 

Ident  intersect,  diverg 

157 

59.2 

36.9 

3.2 

0.6 

2 

90 

7d. 

Superimpose  lines 

6 

0.0 

100.0 

-- 

0.0 

12 

32 

7e. 

Draw  lines* 

145 

-- 

100.0 

-  - 

-- 

-- 

-- 

El-anas 

8a.  Match  plane  shapes 

156 

80.1 

19.9 

0.0 

0.0 

3 

30 

8b. 

Ident  geometric 

47 

100.0 

0.0 

-- 

0.0 

4 

20 

8c. 

Apply  shape  terms 

156 

58.3 

23.1 

9.0 

9.6 

4 

106 

8d. 

Match  patterns 

156 

81.4 

10.9 

3.8 

3.8 

3 

102 

8e. 

Figure  orient 

157 

90.4 

7.6 

-- 

1.9 

3 

30 

Anales  &  Trianales 

9a.  Ident  angles 

162 

16.0 

59.3 

17.3 

7.4 

8 

171 

9b. 

Ident  types  angles 

18 

5.6 

77.8 

-- 

16.7 

12 

85 

9c. 

Ident  types  triangles 

9 

44.4 

55.6 

0.0 

0.0 

4 

36 

9d. 

Ident  altitudes,  bisect 

162 

24.7 

52.5 

21.0 

1.9 

7 

303 

9e. 

Name  angles 

17 

41.2 

58.8 

0.0 

7 

76 

♦Paper  lessons  indicated  by  asterisk;  score  shown  as  100.0  in  "Passed  Short" 
column. 

♦♦Percent  failing  test  after  final  lesson  for  prerequisite  competency;  dashes  in 
"Passed  Long"  column  indicate  no  long  lesson  for  competency. 


36 


Table  3.8.  (Cont.)  Number  and  Percent  of  Soldiers  Attempting  Each  Lesson 
with  Summary  Performance  Statistics  (N- 179) 


Lesson 

Attempt 
Lesson 
(No.) _ 

Passed 

Pretest 

(Pet.) 

Passed 

Short* 

(Pet.) 

Passed 

Long 

(Pel.) 

Failed** 
(Pet.) _ 

Minutes  to 
Complete 
Min  Max 

Solids 

10a.  Match  names/figures 

168 

17.3 

62.5 

18.5 

1.8 

3 

81 

Terminology 

11a.  Ident  shape,  position* 

153 

-- 

100.0 

-- 

-- 

-- 

-- 

lib.  Ident  spatial  terms* 

153 

-  - 

100.0 

-  - 

-- 

-- 

-- 

Addition  &  Sq^tnCtiPH 

12a.  Whole,  no  carry 

167 

99.4 

0.6 

0.0 

0.0 

5 

53 

12b.  Whole,  carry 

167 

89.8 

9.0 

0.6 

0.6 

5 

260 

12c.  Decimals,  carry 

167 

81.4 

13.8 

4.2 

0.6 

4 

271 

12d.  Pos  &  neg  numbers 

164 

13.4 

41.5 

32.9 

12.2 

3 

790 

12e.  24-hour  time 

164 

51.2 

23.2 

13.4 

12.2 

5 

608 

12f.  Increments  measuring 

162 

73.5 

19.1 

6.2 

1.2 

9 

147 

12g.  Linear,  dry,  liquid,  deg 

58 

15.5 

44.8 

22.4 

17.2 

10 

411 

12h.  Estimate  sum,  diff 

162 

1.2 

17.9 

22.2 

58.6 

9 

220 

MultiDl ication  J  Qivi$iqn 

13a.  Whole  numbers 

153 

83.7 

13.1 

3.3 

0.0 

12 

325 

13b.  Whole  &  decimal 

154 

44.8 

31.2 

16.2 

7.8 

5 

277 

13c.  Deci,  divisor,  dividend 

151 

31.1 

20.5 

9.3 

39.1 

10 

242 

13d.  Neg  &  pos  numbers 

24 

29.2 

70.8 

0.0 

0.0 

5 

44 

13e.  Est  product,  quotient 

20 

75.0 

20.0 

0.0 

5.0 

6 

87 

Fractions/Decimal s 

14a.  Est  frac  length,  area, 
vol 

149 

67.1 

18.1 

7.4 

7.4 

4 

168 

14b.  Reduce 

149 

76.5 

18.1 

2.7 

2.7 

4 

409 

14c.  Convert  decs/fracs 

20 

60.0 

20.0 

5.0 

15.0 

8 

118 

14d.  Convert  decimal  s/%' s 

148 

40.5 

53.4 

4.1 

2.0 

5 

228 

14e.  Add,  subtract  fracs 

56 

51.8 

39.3 

7.1 

1.8 

5 

176 

14f.  Multiply,  divide  fracs 

148 

27.7 

49.3 

19.6 

3.4 

10 

311 

14g.  Estimate  fraction 

35 

22.9 

20.0 

25.7 

31.4 

5 

200 

*Paper  lessons  indicated  by  asterisk;  score  shown  as  100.0  in  "Passed  Short" 
column. 

**Percent  failing  test  after  final  lesson  for  prerequisite  competency;  dashes  in 
"Passed  Long"  column  indicate  no  long  lesson  for  competency. 
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Table  3.8.  (Cont.)  Number  and  Percent  of  Soldiers  Attempting  Each  Lesson 
with  Summary  Performance  Statistics  (N-179) 


Less 

on 

Attempt 

Lesson 

(No.) 

Passed 

Pretest 

(Pet.) 

Passed 

Short* 

(Pet.) 

Passed 

Long 

(Pet.) 

Minutes  to 
Failed**  Complete 
(Pet.)  Min  Max 

Geometry 

15a.  Draw  plane  fig 

146 

84.2 

15.8 

0.0 

3 

29 

15b. 

Match  figures/names 

146 

83.6 

14.4 

1.4 

0.7 

3 

58 

15c. 

Label  objects,  figures 

146 

86.3 

13.7 

-- 

0.0 

3 

39 

15d. 

Use  protractor 

143 

82.5 

4.2 

7.7 

5.6 

0 

163 

15f . 

Area,  perimeter  rectang 

37 

2.7 

59.5 

37.8 

0.0 

4 

54 

15g. 

Radius,  area,  circum  cir 

28 

10.7 

82.1 

3.6 

3.6 

9 

171 

15h. 

Measure  rectang  solids 

35 

0.0 

51.4 

31.4 

17.1 

15 

128 

1 5  i . 

Geometric  prob  formulas 

16 

50.0 

43.8 

6.3 

0.0 

10 

222 

15  j . 

Oscilloscope  readouts 

11 

0.0 

9.1 

36.4 

54.5 

58 

264 

Combination  of  Processes 

16a.  Locate  center  of  object 

1 

100.0 

0.0 

0.0 

3 

3 

16b. 

Compute  averages 

144 

64.6 

26.4 

3.5 

5.6 

7 

121 

16c. 

All  proc,  whole/mixed 

23 

17.4 

26.1 

26.1 

30.4 

8 

515 

16d. 

All  proc,  units  meas 

21 

4.8 

19.0 

33.3 

42.9 

11 

121 

16e. 

Info  charts,  graphs,  etc 

.  11 

54.5 

36.4 

9.1 

0.0 

6 

97 

16f . 

Conversion  probs 

41 

29.3 

53.7 

7.3 

9.8 

5 

254 

16g. 

Ratio,  proportion  probs 

143 

31.5 

49.0 

7.0 

12.6 

0 

231 

16h. 

All  proc,  word  probs 

24 

50.0 

33.3 

12.5 

4.2 

5 

76 

Graphing  jp  Saardinals 
Plane 


17a.  Grid  coords  map 

151 

47.0 

40.4 

10.6 

2.0 

6 

310 

17b.  6-digit  coords 

151 

78.1 

17.2 

-- 

4.6 

4 

61 

17c.  Plot  point 

1 

100.0 

0.0 

0.0 

0.0 

6 

6 

17d.  8-digit  coords 

1 

-- 

100.0 

-- 

-- 

25 

25 

Algebra 

18a.  Equations,  1  unknown 

5 

40.0 

0.0 

0.0 

60.0 

20 

196 

18b.  Equivalent  equations 

2 

0.0 

50.0 

0.0 

50.0 

25 

110 

18c.  Calculate  power,  sq  root 

12 

41.7 

58.3 

0.0 

0.0 

9 

47 

Trigonometry 

19a.  Use  trig  tables 

0 

-- 

-- 

-- 

-- 

-- 

19b.  Log  tables* 

2 

-- 

100.0 

-- 

-- 

-- 

-- 

19c.  Trig  solve  geometry 

0 

-- 

-- 

-- 

-- 

-- 

-- 

19d.  Trig  funr  tbls  (degrees) 

0 

♦Paper  lessons  indicated  by  asterisk;  score  shown  as  100.0  in  "Passed  Short" 
column. 

♦♦Percent  failing  test  after  final  lesson  for  prerequisite  competency;  dashes  in 
"Passed  Long"  column  indicate  no  long  lesson  for  competency. 


38 


Table  3.8.  (Cont.)  Number  and  Percent  of  Soldiers  Attempting  Each  Lesson 
with  Summary  Performance  Statistics  (N-179) 


Lesson 

Attempt 
Lesson 
(No.) _ 

Passed 

Pretest 

(Pet.) 

Passed 
Short* 
(Pet  J 

Passed 
Long 
(Pet. ) 

Minutes  to 
Failed**  Complete 
(Pet.)  Min  Max 

Procedural  Directions 

25a.  Follow  directions* 

131 

-- 

100.0 

-- 

-- 

-- 

-- 

25b.  Text,  complete  task* 

131 

-- 

100.0 

-- 

-- 

-- 

-- 

25c.  Follow  details,  seq* 

130 

-- 

100.0 

-- 

-- 

-- 

-- 

25d.  Determine  message* 

129 

-- 

100.0 

-- 

-- 

-- 

-- 

25e.  Select  decision* 

49 

-- 

100.0 

-- 

-- 

-- 

-- 

25f.  Synthesize  info* 

16 

-- 

100.0 

-- 

Vocabulary 

26a.  Common  words 

160 

76.3 

10.0 

13.8 

0.0 

7 

926 

26b.  Aircraft,  tank  words 

6 

83.3 

16.7 

0.0 

0.0 

6 

51 

26c.  Mean  from  context 

157 

93.0 

2.5 

1.9 

2.5 

3 

302 

26d.  Contract, abbrev,  acronym 

>  155 

27.1 

52.9 

13.5 

6.5 

5 

210 

26e.  Figurative,  idiomatic 

155 

80.6 

11.0 

2.6 

5.8 

4 

214 

26f.  Communic,  navig  words 

156 

23.7 

70.5 

-- 

5.8 

5 

325 

26g.  Rifle,  survival  words 

156 

32.1 

60.3 

-- 

7.7 

4 

77 

Reference  Skill? 

27a.  Locate  documents 

131 

99.2 

0.8 

0.0 

*0.0 

3 

22 

27b.  Locate  &  file 

131 

96.2 

3.1 

0.8 

0.0 

4 

38 

27c.  Tbl  cont,  index,  gloss 

130 

81.5 

13.8 

2.3 

2.3 

3 

144 

27d.  Locate  title,  page,  etc. 

130 

95.4 

4.6 

0.0 

0.0 

8 

49 

27e.  Skim,  scan  for  info 

133 

18.8 

42.1 

17.3 

21.8 

4 

176 

27f.  Use  cross  references 

129 

69.0 

19.4 

10.1 

1.6 

5 

172 

Tat?l?s/Chart§ 

28a.  Fact  from  2-col  table 

145 

97.2 

2.8 

0.0 

0.0 

0 

27 

28b.  Fact  from  tbl  intersect 

133 

99.2 

0.8 

-  - 

0.0 

0 

26 

28c.  Complex  tbl,  cross  ref 

69 

47.8 

21.7 

17.4 

13.0 

0 

211 

28d.  Tbls  locate  malfunction 

143 

61.5 

22.4 

10.5 

5.6 

0 

225 

niu$tratipn? 

29a.  Details  from  ill  us 

137 

83.9 

12.4 

3.6 

0.0 

6 

92 

29b.  Details  from  key,  legend 

137 

98.5 

1.5 

0.0 

0.0 

5 

28 

29c.  Use  cross-sec  view 

137 

99.3 

0.7 

0.0 

0.0 

4 

29 

29d.  3-0  projection 

137 

75.9 

18.2 

-- 

5.8 

3 

39 

29e.  Illus  to  follow  direct 

139 

99.3 

0.7 

0.0 

0.0 

5 

23 

♦Paper  lessons  indicated  by  asterisk;  score  shown  as  100.0  in  "Passed  Short" 
column. 

♦♦Percent  failing  test  after  final  lesson  for  prerequisite  competency;  dashes  in 
"Passed  Long"  column  indicate  no  long  lesson  for  competency. 
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Table  3.8.  (Cont.)  Number  and  Percent  of  Soldiers  Attempting  Each  Lesson 
with  Summary  Performance  Statistics  (N=179) 


Attempt 

Passed 

Passed 

Passed 

Minutes  to 

Lesson 

Pretest 

Short* 

Long 

Failed** 

Complete 

mtsmm 

lism 

mismu 

uLRmim 

Min 

Max 

Flow 

Charts 

30a. 

Meanings  flow  cht  symbol 

140 

3.6 

95.7 

-- 

0.7 

2 

80 

30b. 

Flow  cht,  proced  decis 

8 

50.0 

37.5 

-- 

12.5 

6 

38 

30c. 

Ident  organ iz  members 

4 

100.0 

0.0 

0.0 

5 

7 

Schematics 

31a. 

Locate  subsys  block 

26 

0.0 

61.5 

26.9 

11.5 

15 

137 

31b. 

Components,  signal  path 

21 

0.0 

85.7 

14.3 

0.0 

35 

157 

31c. 

Circuit  connects  schema 

18 

33.3 

44.4 

-- 

22.2 

4 

37 

3  Id . 

Faulty  compon/trbl shoot 

4 

100.0 

0.0 

0.0 

0.0 

6 

9 

31e. 

Symbol:  comp,  signal  path  6 

83.3 

16.7 

0.0 

0.0 

4 

23 

Forms 

32a. 

Locate  block  for  info 

133 

99.2 

0.8 

0.0 

0.0 

3 

32 

32b. 

Transfer  data 

131 

99.2 

0.8 

0.0 

0.0 

5 

26 

32c. 

Enter  info  on  form 

129 

77.5 

13.2 

2.3 

7.0 

8 

225 

32e. 

Use  form  to  find  info 

129 

99.2 

0.8 

0.0 

0.0 

4 

28 

Note-Takina 

33a. 

Record  info* 

123 

-  - 

100.0 

-- 

-  - 

33b. 

Accuracy  recording* 

123 

-- 

100.0 

-- 

-- 

33c. 

Record  info  sentence* 

5 

-- 

100.0 

-- 

-  - 

-  - 

-  - 

33d. 

Record  info  >1  sentence* 

1 

-- 

100.0 

-- 

-- 

-- 

-- 

Outlininq  (ToDic  or  Sentence! 

34a. 

Ident  main  idea 

5 

0.0 

20.0 

80.0 

11 

45 

34b. 

Titles  for  outline 

3 

100.0 

0.0 

0.0 

6 

8 

34c. 

Select  details 

5 

100.0 

0.0 

0.0 

9 

16 

34d. 

Label  topics 

131 

7.6 

46.6 

-- 

45.8 

26 

336 

34e. 

Training  out! ine* 

117 

100.0 

*Paper  lessons  indicated  by  asterisk;  score  shown  as  100.0  in  "Passed  Short" 
column. 

**Percent  failing  test  after  final  lesson  for  prerequisite  competency;  dashes  in 
"Passed  Long"  column  indicate  no  long  lesson  for  competency. 
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Table  3.8.  (Cont.)  Number  and  Percent  of  Soldiers  Attempting  Each  Lesson 
with  Summary  Performance  Statistics  (N*179) 


Lesson 

Attempt 
Lesson 
(No.) _ 

Passed 

Pretest 

(Pet.) 

Passed 

Short* 

(Pet.) 

Passed 

Long 

(Pet.) 

Minutes  to 
Failed**  Complete 
(Pet.)  Min  Max 

ReDort  Writing 

35a.  Ident  objectives* 

22 

-- 

100.0 

-  - 

-  - 

-  - 

35b.  Who,  what,  when,  etc.* 

116 

100.0 

-  - 

-  - 

35c.  Relevant  details* 

23 

100.0 

-  - 

•  - 

35d.  Sequence  events* 

115 

-- 

100.0 

-- 

•  - 

35e.  Impressions  of  event* 

115 

-- 

100.0 

-  - 

«  - 

35f.  Write  report* 

114 

-- 

100.0 

-- 

-- 

-  - 

•  - 

35g.  Summarize  events* 

0 

-- 

-- 

-  - 

-  - 

-  - 

•  - 

35h.  Major  points* 

114 

-- 

100.0 

-  - 

-  - 

-  - 

•  - 

35i.  Justify  action* 

23 

-- 

100.0 

-  - 

-  - 

-  - 

_  . 

35 j.  Report  by  format* 

5 

-- 

100.0 

-- 

-- 

-- 

-- 

Editing 

36a.  Spell  common  words 

47 

59.6 

34.0 

4.3 

2.1 

8 

207 

36b.  Spell  task  words 

44 

81.8 

9.1 

9.1 

0.0 

8 

245 

36c.  Capitalization 

35 

5.7 

34.3 

17.1 

42.9 

8 

263 

36d.  Correct  misspell ing 

28 

78.6 

14.3 

7.1 

0.0 

9 

127 

36e.  Punctuation 

35 

2.9 

25.7 

17.1 

54.3 

50 

453 

36f.  Grammar  rules 

35 

57.1 

40.0 

2.9 

0.0 

8 

160 

36g.  Rewrite  paragraph 

3 

33.3 

66.7 

-  - 

0.0 

8 

49 

36h.  Appraise  &  adjust 

28 

64.3 

25.0 

-- 

10.7 

6 

93 

Precautions 

40a.  Knowledge  prevent  injury 

133 

100.0 

0.0 

0.0 

4 

25 

40b.  Safety/security  probs 

133 

82.7 

15.8 

-  - 

1.5 

3 

72 

40c.  Emergency  action 

135 

44.4 

45.2 

-- 

10.4 

4 

68 

Recognition 

41a.  Label  objects 

133 

98.5 

1.5 

0.0 

0.0 

3 

18 

41b.  Hand  4  arm  signals 

130 

12.3 

76.2 

10.8 

0.8 

4 

235 

41c.  Equip  damage,  defects 

129 

76.0 

22.5 

1.6 

0.0 

6 

186 

41d.  Move,  aline,  connect 

129 

90.7 

7.8 

-  - 

1.6 

2 

17 

41e.  Objs  size,  shape,  etc. 

131 

98.5 

1.5 

0.0 

0.0 

3 

24 

41g.  Choose  action 

128 

72.7 

14.1 

-  - 

13.3 

3 

36 

41h.  Use  symbols  4  codes 

128 

91.4 

6.3 

1.6 

0.8 

6 

110 

*Paper  lessons  indicated  by  asterisk;  score  shown  as  100.0  in  "Passed  Short" 
column. 

♦♦Percent  failing  test  after  final  lesson  for  prerequisite  competency;  dashes  in 
"Passed  Long"  column  indicate  no  long  lesson  for  competency. 
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which  ranges  from  175  for  lessons  such  as  "la.  Match  numeral  with  name"  to  one 
(example,  "17c.  Plot  point,  distance  and  direction  given").  It  also  shows,  based  on  the 
number  attempting  a  lesson,  the  percent  of  those  who: 

-  passed  the  pretest; 

-  failed  the  pretest  but  passed  the  short  lesson  test; 

-  failed  the  short  lesson  test  but  passed  the  long  lesson  test; 

*  failed  the  lesson  (failed  the  long  lesson  test  or  failed  the  short  lesson  test  and  no 
long  lesson  was  available). 

Percentages  allow  easily  understood  comparisons  across  lessons.  They  should  be 
interpreted  cautiously,  however,  for  those  lessons  that  were  attempted  by  only  a  small 
number  of  persons. 

The  last  information  in  Table  3.8  is  the  minimum  and  maximum  amounts  of  time  in 
minutes  to  complete  the  lesson.  These  figures  illustrate  a  problem  in  deciding  how  long  it 
takes  a  soldier  to  complete  JSEP.  For  Lesson  la,  for  example,  the  quickest  soldier  took  8 
minutes  to  complete  the  lesson  (presumably  passing  the  pretest)  and  the  slowest  took  221 
minutes.  In  several  lessons  the  maximum  value  was  difficult  to  accept;  for  example, 
almost  10  hours  for  Lesson  lc,  or  13  hours  for  Lesson  12d.  However,  JSEP’s  management 
system  signs  students  off  from  a  lesson  if  they  go  beyond  a  certain  length  of  time 
without  making  a  response,  so  the  records  should  not  have  included  any  great  periods  of 
time  when  a  student  was  away  from  the  terminal.  Some  of  the  time  could  have  been  used 
by  soldiers  to  write  on-line  notes  about  the  lessons.  There  is  no  way  to  separate  this 
time  from  that  used  in  testing  or  instruction.  At  the  other  extreme,  students  were 
reported  to  have  completed  several  lessons  (e.g.,  15d,  28a)  in  0  minutes.  Here  it  seems 
possible  that  an  instructor  might  have  intervened  to  pass  a  student  manually.  Since  there 
is  no  ready  mechanical  explanation  of  these  extreme  times  we  simply  report  what  was 
recorded,  and  assume  that  the  figures  do  accurately  illustrate  that  some  students  moved 
through  JSEP  at  a  very  deliberate  speed.  And,  since  JSEP  is  a  self-paced  program  it  did 
not  seem  appropriate  to  exclude  data  on  the  grounds  that  a  soldier  was  unusually  slow  or 
fast.  The  best  compromise  we  could  reach  was  that  of  showing  the  median  time  values 
for  soldiers  who  pretested  out  of  a  lesson,  passed  after  the  short  lesson,  and  so  forth. 
These  figures  appear  in  Appendix  C,  Table  C.2. 
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Table  3.8  reinforces  the  instructors’  comments  that  the  JSEP  diagnostic  test  is 
needed  to  save  soldiers  the  time  of  taking  lessons  on  content  they  already  know.  Of  the 
total  186  lessons  in  JSEP,  soldiers  attempted  153  on-line  lessons  during  the  evaluation. 

For  91  of  these  lessons  (59%)  half  or  more  of  the  soldiers  attempting  them  passed  the 
pretest  and  were  thus  exempted  from  the  lesson.  At  the  other  extreme,  18  of  the  lessons 
(12%)  were  failed  after  instruction  by  one-fourth  or  more  of  the  soldiers  attempting  them. 
It  was  not  always  clear  whether  test  performance  reflected  difficult  content  or  difficult 
test  items.  For  example,  in  lesson  "14f.  Multiply  and  divide  fractions,"  about  28  percent 
passed  the  pretest,  49  percent  passed  the  short  lesson  test,  and  3  percent  failed  the 
lesson.  These  seem  plausible  results  for  a  mathematics  skill  that  is  rarely  used  after 
being  taught  in  the  elementary  grades,  but  which  can  often  be  recalled  with  a  little 
practice.  However,  only  3  percent  of  the  soldiers  passed  the  pretest  for  lesson  "lc. 

Identify  number  before,  between"  others,  and  47  percent  failed  the  lesson.  This  seems  a 
likely  instance  of  test  items  that  are  worded  or  scored  in  such  a  way  that  they  are  more 
difficult  than  the  content  they  measure. 

Specific  Lesson  Level  Findings 

Detailed  lesson  performance  data  were  considered  of  limited  interest  to  the  general 
reader,  and  therefore  are  presented  in  Appendix  C,  with  accompanying  text.  Table  C.l 
breaks  out  the  number  of  soldiers  failing  the  pretest  for  each  lesson  and,  among  those, 
the  percent  passing  the  short  lesson  test,  passing  the  long  lesson  test,  or  failing  the 
lesson.  The  same  appendix  includes  Table  C.2,  which  reports  the  median  time  required  to 
complete  each  lesson  by  soldiers  passing  the  pretest,  taking  only  the  short  lesson,  and 
taking  both  the  short  and  long  lessons.  This  analysis  illustrates  the  variation  in  the 
amount  of  time  different  soldiers  required  to  complete  the  same  lesson,  and  demonstrates 
that  considerable  learner  time  was  saved  through  the  pretest  option  that  could  exempt  a 
soldier  from  instruction  in  a  competency  already  mastered. 

Common  Core  Performance 

Every  soldier  may  have  a  different  MOS-based  or  customized  prescription  on  JSEP, 
and  may  work  through  this  list  of  lessons  at  an  individual  pace.  The  one  commonality 
across  all  participants  is  the  combination  of  lessons  comprising  the  Common  Core 
Prescription,  which  is  assigned  to  all  soldiers.  These  122  lessons  (98  of  which  are  on- 
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line)  are  embedded  in  every  MOS  prescription  and  are  the  default  prescription  for  any 
student  whose  primary  or  secondary  MOS  does  not  have  a  JSEP  prescription. 

Table  3.9  displays  an  analysis  of  performance  on  the  98  on-line  Common  Core  lessons 
by  the  evaluation  soldiers.  The  information  is  broken  out  by  lesson  series.  The  number 
of  soldiers  attempting  each  lesson  was  not  necessarily  the  same;  to  create  the  table  we 
added  the  percent  of  soldiers  in  each  status  category  (e.g.,  passing  the  pretest)  for  the 
Common  Core  lessons  of  the  series  and  divided  the  sum  by  the  number  of  Common  Core 
lessons.  The  median  time  to  complete  was  calculated  in  the  same  manner.  The  results 
give  a  very  simplified  picture  of  the  "average"  performance  of  a  hypothetical  Common 
Core  lessons  participant. 

As  might  be  expected  from  the  title,  the  Common  Core  lessons  appear  relatively  easy. 
Table  3.9  shows  that  across  the  63  quantitative  lessons,  soldiers  passed  an  average  of  57 
percent  of  the  lessons  on  the  pretest  and  an  additional  27  percent  on  the  short  lesson 
test.  In  8  percent  of  the  lessons,  on  an  average,  the  soldiers  were  unsuccessful.  Results 
were  similar  for  the  35  verbal  lessons  in  the  Common  Core.  Here,  76  percent  of  the 
lessons  were  passed  on  pretest  and  19  percent  on  the  short  lesson  test.  On  the  average, 
only  3  percent  of  these  lessons  were  failed. 

When  the  averages  are  computed  across  all  lessons,  soldiers  passed  the  pretest  and 
were  exempted  from  taking  64  percent  of  the  lessons.  They  passed  an  additional  24 
percent  with  the  short  lesson,  and  on  the  average  failed  6  percent  of  the  lessons.  The 
median  time  to  complete  the  Common  Core  prescription  was  42  hours  and  57  minutes. 


JSEP  Performance  and  Entry  Achievement 

We  noted  at  the  beginning  of  this  chapter  that  the  soldiers  participating  in  the 
evaluation  had  relatively  high  reading  and  arithmetic  skills  as  reported  on  the  TABE.  In 
fact,  the  average  participant  entered  the  program  with  a  reading  score  almost  equivalent 
to  the  end-of-program  objective  for  BSEP  II.  This  limited  the  extent  to  which  the 
findings  of  the  evaluation  can  be  generalized  to  the  larger  population  of  BSEP-eligible 
soldiers.  Table  3.10  reports  an  analysis  directed  at  this  limitation,  the  examination  of 
soldiers’  performance  on  JSEP  in  light  of  their  entering  TABE  and  GT  scores. 
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Table  3.9.  Soldiers'  Performance  on  Common  Core  Lessons, 

by  Series  (N-179) 


Series* 

Percent 

Pretest 

Passing: 

Short 

Lesson 

Not 

Pass 

Numbering  and  Counting  (9) 

40.9 

29.2 

11.3 

18.6 

Linear,  Weight,  Volume 

Measures  (6) 

60.2 

29.6 

3.8 

6.4 

Time-Telling  Measures  (4) 

35.9 

34.8 

19.0 

10.3 

Gage  Measures  (9) 

71.7 

22.2 

4.6 

1.5 

Spatial  (4) 

68.9 

19.2 

4.1 

8.0 

Lines  (3) 

53.7 

41.1 

4.0 

0.8 

Planes  (4) 

77.6 

15.4 

3.2 

3.8 

Angles  &  Triangles  (2) 

20.4 

55.9 

19.2 

4.7 

Solids  (1) 

17.3 

62.5 

18.5 

1.8 

Addition  &  Subtraction  (7) 

58.6 

17.9 

11.4 

12.2 

Multiplication  4  Division  (3) 

53.3 

21.6 

9.6 

15.6 

Fractions  4  Decimals  (4) 

53.0 

34.7 

8.5 

3.9 

Geometry  (4) 

84.2 

12.0 

2.3 

1.6 

Combination  of  Processes  (2) 

64.6 

26.4 

3.5 

5.6 

Graphing  in  the  Coordinate 
Plane  (1) 

47.0 

40.4 

10.6 

2.0 

Subtotal;  Quantitative 

Lessons  (63) 

57.4 

26.8 

8.1 

7.6 

Vocabulary  (6) 

55.5 

34.5 

5.3 

4.7 

Reference  Skills  (6) 

78.2 

12.9 

4.1 

4.8 

Tables  4  Charts  (3) 

86.0 

8.7 

3.5 

1.9 

Illustrations  (5) 

91.4 

6.7 

0.7 

1.2 

Flow  Charts  (I) 

3.6 

95.7 

0.0 

0.7 

Forms  (4) 

93.8 

3.9 

0.6 

1.8 

Precautions  (3) 

75.7 

20.3 

0.0 

4.0 

Recognition  (7) 

77.2 

18.6 

2.0 

2.4 

Subtotal :  Verbal 

Lessons(35) 

76.1 

18.5 

2.5 

3.0 

Total:  All  Lessons  (98) 

64.1 

23.8 

6.1 

6.0 

Median  Time  to  Complete  Common 

Core  Lessons:  42  hours 

57  minutes 

♦Average  for  Series;  number  on-line  common  core  lessons  shown  in 
parentheses  for  each  Series.  Not  all  soldiers  attempted  all 
lessons. 
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Table  3.10.  Percent  of  Lessons  Not  Needed  or  Failed  Among 
Soldiers  at  Various  Entry  Achievement  Levels  (N-179) 


Soldiers  Scorina 

Average  Percent 
Not  Needed 

Ouant  Verbal 

Lessons: 

All 

Failed 

Ouant 

Verbal 

All 

TABE  Grade  Ranges 

Reading  <8.9  (35)* 

44.5 

62.1 

49.4 

11.6 

7.6 

10.3 

Reading  9.0--10.9  (70) 

52.6 

69.8 

57.4 

8.2 

4.4 

7.6 

Reading  >11.0  (30) 

62.5 

80.1 

68.0 

5.3 

2.4 

4.5 

Math  <6.9  (13) 

39.9 

59.9 

47.4 

20.2 

12.1 

16.8 

Math  7.0--8.9  (70) 

49.5 

69.8 

53.7 

9.4 

4.5 

8.6 

Math  >9.0  (52) 

60.2 

73.4 

65.5 

4.1 

3.2 

3.9 

8SEP  TABE  Goal 

Reading  <10.1  (111) 

47.9 

68.5 

53.6 

10.4 

5.8 

9.2 

Reading  >10.1  (68) 

58.1 

74.7 

63.2 

6.3 

2.7 

5.6 

Math  <10.1  (160) 

50.3 

70.6 

55.8 

9.6 

4.8 

8.4 

Math  >10.1  (19) 

64.2 

74.0 

69.9 

2.4 

3.4 

2.9 

GT  Score  Ranges 

GT  <89  (57) 

47.9 

65.6 

52.6 

10.2 

5.1 

9.1 

GT  90--99  (82) 

52.5 

72.3 

58.3 

8.5 

6.7 

7.8 

GT  >100  (32) 

57.9 

76.8 

62.9 

6.9 

2.2 

6.0 

♦Number  of  observations  shown  in  parentheses. 
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Across  subject  areas  and  achievement  measures,  the  higher  the  soldiers’  scores,  the 
greater  the  proportion  of  JSEP  lessons  they  did  not  need  (i.e.,  passed  the  pretest  for)  and 
the  smaller  the  proportion  of  lessons  they  failed  after  instruction.  For  example,  soldiers 
scoring  below  the  9.0  grade  level  in  reading  on  the  TABE  pretested  out  of  half  (49%)  of 
their  JSEP  lessons;  those  scoring  at  the  11.0  grade  level  or  above  in  reading  passed  the 
pretest  for  more  than  two-thirds  (68%)  of  their  lessons.  Higher  scores  in  reading  on  the 
TABE  were  associated  with  greater  success  in  both  quantitative  and  verbal  lessons  on 
JSEP;  the  same  was  true  for  TABE  mathematics  achievement. 

When  the  data  were  sorted  to  separate  soldiers  who  had  met  the  10.1  grade  level 
criteria  in  reading  or  mathematics  that  are  the  objectives  for  BSEP  II  from  those  who  had 
not,  the  findings  were  similar.  Among  soldiers  scoring  at  the  10.1  grade  level  or  better 
on  the  TABE  in  either  subject,  about  two-thirds  of  the  JSEP  lessons  were  passed  on 
pretest  and  no  further  instruction  was  needed.  And,  these  soldiers  failed  relatively  few 
lessons. 

Entry  GT  score  was  also  associated  with  JSEP  lesson  performance.  The  soldiers  who 
began  JSEP  with  a  GT  score  of  100  or  more  tested  out  of  three-fifths  (58%)  of  the 
quantitative  and  three-fourths  (77%)  of  the  verbal  lessons.  They  failed  about  7  percent  of 
the  quantitative  and  2  percent  of  the  verbal  lessons. 

Performance  Standards 

There  are  no  objective  standards  for  how  many  lessons  a  soldier  should  pass  on 
pretest  (and  thus  not  need)  or  of  how  many  a  soldier  can  fail  acceptably.  With  no 
operational  definition  of  "when  JSEP  works,"  it  is  impossible  to  specify  which  soldiers  it 
works  for.  Soldiers  meeting  BSEP  graduation  goals  on  the  TABE  (10.1  grade  level),  or  the 
general  reenlistment  criterion  of  100  on  the  GT,  passed  an  average  of  two-thirds  of  the 
JSEP  lessons  on  the  pretest.  Whether  JSEP  is  suitable  or  too  easy  for  this  group  is  a 
matter  of  instructional  judgment.  On  the  other  hand,  soldiers  in  the  lowest  ability 
groupings  reported  here  were  typically  unsuccessful  in  about  10  percent  of  the  JSEP 
lessons  they  attempted.  Those  scoring  below  the  7.0  grade  level  in  mathematics  failed 
almost  twice  as  many  quantitative  lessons  as  any  other  group.  Deciding  whether  a 
soldier’s  achievement  level  is  too  low  for  him  or  her  to  profit  from  JSEP  instruction  is 
also  a  matter  of  judgment.  Finally,  we  must  point  out  again  that  the  entry  achievement 
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level  of  these  soldiers  was  higher  than  is  typical  for  BSEP  classes.  The  sample  may  not 
have  included  soldiers  who  had  reading  and  math  skills  poor  enough  to  prevent  them  from 
learning  from  the  program;  how  frequently  such  soldiers  will  be  referred  to  JSEP  is  an 
unanswered  question. 


Completion  Rates 

JSEP  is  designed  as  an  open-entry,  open-exit  program,  although  during  the  evaluation 
period  soldiers  were  typically  released  from  their  regular  duties  for  a  prespecified  number 
of  weeks.  Table  3.11  reports  information  from  the  logs  maintained  by  the  JSEP 
instructors  concerning  completion  rates,  and  reflects  all  evaluation  sites.  The  table 
suggests  that  most  soldiers  do  finish  the  program.  At  the  end  of  the  evaluation  period, 

63  percent  of  the  participants  had  completed  JSEP,  and  another  15  percent  were  either 
still  enrolled  or  had  left  temporarily  and  were  expected  to  return.  In  only  a  few  cases 
(4%)  had  a  soldier  dropped  voluntarily  or  been  dismissed  from  the  program  for  reasons 
such  as  excessive  absenteeism.  Completion  status  data  were  missing  for  18  percent  of  the 
group.  However,  the  findings  here  agree  with  those  from  interviews  with  participants  and 
instructors,  which  are  reported  in  Chapter  4.  Soldiers,  with  few  exceptions,  like  JSEP  and 
persist  in  it.  The  fact  that  some  soldiers  were  expected  to  return  after  a  short  break  in 
their  participation  suggests  that  the  program  does  have  the  scheduling  flexibility  i:  was 
designed  to  offer. 


Table  3.11.  Reasons  for  Leaving  JSEP-- 
All  Soldiers  (N-207) 


Reason 

Number 

Percent 

Completed  JSEP 

130 

62.8 

Not  completed  (still  enrolled, 
temporary  break  in  enrollment) 

31 

15.0 

Dropped  or  removed  from  JSEP 

8 

3.9 

Missing  data 

38 

18.4 
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Informal  Foilow-Up  Data 


The  evaluation  was  limited  to  90  days  and  did  not  include  any  formal  measurement  of 
performance  beyond  the  GAP  posttest  administered  immediately  upon  finishing  JSEP. 
However,  the  instructors  collected  some  follow-up  information  on  their  students,  and 
forwarded  this  to  us.  While  these  data  were  not  collected  as  part  of  the  evaluation,  and 
we  cannot  claim  credit  or  responsibility  for  them,  they  do  give  some  idea  of  JSEP’s 
effects  in  a  realistic  setting.  Not  every  evaluation  site  collected  these  data. 

Some  62  soldiers  were  posttested  on  the  TABE  as  soon  as  they  finished  JSEP,  and 
the  results  are  shown  in  Table  3.12.  When  the  results  are  compared  with  the  TABE 
scores  acquired  at  some  time  prior  to  JSEP,  the  group  showed  an  average  gain  of  0.6 
grade  levels,  from  10.2  to  10.8,  in  reading.  Before  JSEP,  25  percent  were  reading  at  a 
grade  11.0  level  or  higher;  after  JSEP,  44  percent  performed  at  this  standard.  In 
mathematics  the  increase  was  1.6  grade  levels  on  an  average,  from  8.9  to  10-5.  Some  10 
percent  scored  at  the  11th  grade  level  or  higher  before  JSEP,  and  the  figure  increased  to 
42  percent  after  JSEP.  The  instructors  from  the  two  sites  providing  TABE  data  reported 


Table  3.12.  Performance  on  Tests  of  Adult  Basic  Education  (TABE) 
by  Soldiers  at  Two  JSEP  Sites  (N=*62) 


Total  Reading:  Total  Math: 

Pretest  Posttest  Pretest  Posttest 


Grade  Level 

No. 

Pet. 

No. 

Pet. 

No. 

Pet. 

No. 

Pet. 

Grade  5- - 5 . 9 

.. 

1 

1.6 

Grade  6--6.9 

-- 

-- 

1 

1.6 

5 

8.1 

1 

1.6 

Grade  7--7.9 

3 

4.8 

-- 

-- 

13 

21.0 

4 

6.5 

Grade  8--8.2 

10 

16.1 

3 

4.8 

18 

29.0 

3 

4.8 

Grade  9- - 9 . 9 

15 

24.2 

11 

17.7 

10 

16.1 

16 

25.8 

Grade  10--10.9 

18 

29.0 

20 

32.3 

10 

16.1 

11 

17.7 

Grade  11-11.9 

5 

8.1 

15 

24.2 

4 

6.5 

10 

16.1 

Grade  12-12.9 

11 

17.7 

12 

19.4 

2 

3.2 

16 

25.8 

Average  Score 

10.2 

10.8 

8.9 

10.5 

Average  Gain 

0.6 

1.6 
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that  there  was  no  additional  instruction  between  completion  of  JSEP  and  administration  of 
the  TABE.  We  do  not  know  what  formal  instruction  or  other  learning  opportunities  may 
have  occurred  between  the  initial  TABE  and  the  time  at  which  the  soldiers  began  JSEP. 
Parenthetically,  the  average  pre-JSEP  TABE  scores  for  this  group  were  about  0.2  grade 
levels  higher  than  those  for  the  JSEP  soldiers  as  a  whole. 

GT  performance  for  69  soldiers  at  five  of  the  evaluation  sites  is  shown  in  Table  3.13. 
Among  these  JSEP  participants,  an  average  pre-JSEP  GT  score  of  94.8  increased  to  an 
average  post-JSEP  score  of  108 .5;  an  increase  of  13.7  points.  Perhaps  more  importantly, 

84  percent  of  the  post-JSEP  group  achieved  a  score  of  100  or  better,  compared  to  19 
percent  of  the  group  before  JSEP. 


Table  3.13.  Performance  on  General  Technical  Composite  (GT) 
by  Soldiers  at  Five  JSEP  Sites  (N-69) 


Pretest  Posttest* 


Score  Ranoe 

No. 

Pet. 

No. 

Pet. 

110  or  above 

37 

53.6 

100  -  109 

13 

18.8 

21 

30.4 

90  -  99 

39 

56.5 

10 

14.5 

80  -  89 

15 

21.7 

1 

1.5 

Less  than  80 

2 

2.9 

-- 

Average  Score 

94.8 

108.5 

Average  Gain 

13.7 

♦Instructors  reported  that  "all  but  a  few"  soldiers  received  BSEP  or 
other  math  or  language  arts  instruction  between  JSEP  and  posttest, 
but  did  not  identify  soldiers. 
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Two  caveats  need  to  be  made  in  regard  to  these  findings.  The  first  is  that  the 
instructors  reported  that  all  but  a  few  soldiers  had  received  additional  GT  preparation 
between  the  time  they  completed  JSEP  and  the  time  they  took  the  GT  (they  did  not 
identify  which  soldiers  had  received  supplementary  work).  This  preparation  varied  by  site, 
but  included  drill  in  such  things  as  GT  mathematics,  vocabulary,  and  taking  timed  tests. 
The  second  warning  is  that  selection  may  have  taken  place.  It  seems  plausible  that  less 
prepared  soldiers  might  have  been  discouraged  from  attempting  the  GT.  We  have  no  way 
of  knowing  if  this  occurred.  An  earlier  report  (Hahn  et  al.,  1986)  found  average  GT  gains 
of  about  16  points  following  BSEP  II  and  an  average  gain  of  10  points  with  an  early 
version  of  McFann*Gray.  This  suggests  that  if  JSEP  is  coupled  with  supplementary 
instruction  it  is  about  as  effective  as  BSEP  in  improving  GT,  but  that  BSEP  is  the 
program  of  choice  for  GT  development. 


Summary 

The  preceding  pages  have  presented  a  great  deal  of  information  addressing  a  number 
of  related,  but  still  different,  areas.  This  section  summarizes  the  findings  briefly  in  order 
to  draw  a  coherent  picture  of  what  they  may  mean  when  taken  as  a  whole.  It  should  be 
repeated  that,  because  of  limitations  in  sample  size  and  the  data  that  could  be  collected, 
the  analysis  was  largely  restricted  to  the  performance  of  soldiers  in  on-line  lessons  at 
four  PLATO  sites. 

Completing  a  prescription  of  JSEP  lessons  is  associated  with  small  gains  (about  3 
points)  on  a  test  designed  to  parallel  the  GT  composite.  Among  the  approximately  one- 
third  of  the  soldiers  who  received  some  additional  GT  preparation  and  then  went  on  to 
take  that  test,  GT  gains  were  about  as  great  as  those  reported  for  BSEP  in  earlier 
evaluations.  In  analyzing  the  GAP  Test  results,  a  student’s  entry  score  was  the  strongest 
predictor  of  exit  score.  A  soldier’s  rank,  years  of  experience  in  the  Army,  and  level  of 
civilian  education  had  little  relation  to  final  score.  Number  of  lessons  and  time  in  JSEP 
did,  at  least  in  mathematics:  the  faster  a  soldier  moved  through  JSEP,  the  higher  his  or 
her  final  test  performance  was. 

At  the  level  of  individual  lessons,  soldiers  pretested  out  of  about  60  percent  of  the 
lessons  assigned  to  them.  Among  the  lessons  for  which  soldiers  failed  the  pretest,  about 
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20  percent  were  failed  by  one-fourth  or  more  of  the  soldiers  after  instruction.  When  we 
looked  at  the  98  on-line  Common  Core  lessons,  which  are  embedded  in  every  prescription, 
we  concluded  (very  roughly)  that  the  lessons  were  needed  by  and  effective  for  about  one- 
third  of  the  soldiers  (excluding  the  64  percent  who  passed  the  pretests  and  the  6  percent 
who  failed  the  lessons)  for  whom  they  were  prescribed. 

The  amount  of  time  soldiers  needed  to  complete  a  prescription,  or  even  a  lesson, 
varied  so  much  that  it  made  little  sense  to  talk  about  averages.  For  the  98  on-line 
lessons  of  the  Common  Core  (part  of  every  MOS  prescription)  the  median  time  for 
completion  was  about  43  hours.  The  Common  Core  is  also  the  prescription  for  soldiers 
when  there  is  no  JSEP  prescription  matching  their  primary  or  secondary  MOS.  There  was 
a  JSEP  prescription  for  the  primary  MOS  of  about  70  percent  of  the  soldiers;  the 
proportion  was  the  same  for  the  soldiers’  secondary  MOSs. 

Finally,  we  must  note  again  that  the  entry  achievement  levels  of  the  soldiers  in  this 
evaluation  were  high,  particularly  in  reading.  The  cross  tabulation  of  lesson  performance 
by  entry  TABE  and  GT  scores  demonstrates  that  soldiers  with  higher  achievement  levels 
pretest  out  of  more,  and  fail  fewer,  JSEP  lessons.  We  cannot  say  how  "typical  BSEP" 
soldiers  would  perform  in  JSEP.  Indications  are  that  they  would  find  it  more  salient,  but 
also  more  difficult,  than  the  soldiers  in  this  sample  did. 

The  next  chapter  describes  JSEP  as  it  was  understood  through  interviews  with,  and 
observations  of,  the  soldiers  and  educational  staff  persons  participating  in  the  program. 
This  information  suggests  possible  explanations  for  the  performance  data  just  reported. 
That  chapter  is  followed  by  the  major  findings  and  conclusions  emerging  from  the 
evaluation  as  a  whole. 
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Chapter  4.  Site  Visits  and  Surveys:  Descriptions 
of  JSEP 


This  chapter  presents  the  information  collected  by  observing  and  talking  with 
soldiers  and  educators  as  they  used  JSEP,  complemented  by  the  subjective  evaluations 
these  participants  reported  later  through  attitude  surveys.  Visits  to  the  JSEP  pilot  sites 
comprised  a  major  activity  in  the  evaluation.  These  site  visits  took  a  naturalistic  position 
toward  evaluation,  and  yielded  a  different  kind  of  information  than  that  collected  with  the 
tests  and  records  used  in  other  evaluation  activities.  The  major  purpose  of  the  site  visits 
was  to  discover  how  JSEP  operated  in  its  natural  setting  of  an  Army  Education  Center. 
This  in  turn  broadened  our  understanding  of  the  other  evaluation  findings.  We  were 
interested  here  in  expanding  our  understanding  of  JSEP  rather  than  testing  predetermined 
hypotheses,  in  discovering  findings  that  were  relevant  to  program  operation,  and  in  using 
observation  and  interview  techniques  that  did  not  limit  respondents  to  the  choices  decided 
beforehand  by  outside  researchers  (Guba  and  Lincoln,  1981).  Mixing  the  naturalistic  and 
traditional  evaluation  paradigms  is  particularly  useful  in  evaluating  untried  or  innovative 
programs.  The  approach  helps  to  reassure  us  that  important  historical  or  contextual 
factors  have  not  been  ignored.  It  prevents  us  from  repeating  the  mistake  of  the  blind 
sages  in  the  fable,  of  letting  our  expectations  about  ropes  and  umbrellas  keep  us  from 
perceiving  the  elephant  that  is  actually  there. 

Conducting  the  Site  Visits 

Six  of  the  seven  installations  piloting  JSEP  were  visited  between  mid-February  and 
mid-April  1988.  The  seventh  pilot  site,  Walter  Reed  Army  Medical  Center,  did  not  begin 
using  JSEP  until  the  middle  of  February.  Because  experiences  here  differed  so  much  from 
those  at  other  sites,  Walter  Reed  was  not  included  in  this  set  of  site  visits.  Instead,  the 
staff  arranged  for  an  evaluator  to  visit  informally  a  number  of  times. 


All  of  the  six  site  visits  were  structured  in  the  same  manner.  Following  a  request 
for  access  to  the  JSEP  programs  sent  to  the  installations  by  the  Soldier  Education 
Division,  we  contacted  each  JSEP  Coordinator  to  reach  an  agreement  about  the  time  and 
activities  of  the  visit.  We  explained  that  we  wanted  to  observe  JSEP  classes  and  to  talk 
with  students  and  ail  education  or  training  staff  members  whose  work  involved  JSEP.  The 
JSEP  Coordinator  was  also  asked  to  set  up  interviews  with  past  JSEP  students  and  with 
several  supervisors  of  current  or  past  students.  The  typical  procedure  was  to  schedule 
appointments  with  supervisors  and  JSEP  graduates  and  to  talk  with  others  who  were 
routinely  in  the  Education  Center  whenever  convenient.  A  team  of  two  evaluators  visited 
each  installation,  spending  two  and  one-half  days  on  site.  Interviews  were  usually  one- 
on-one,  but  in  a  few  instances  we  talked  with  a  small  group  of  students  (for  example,  in 
a  break  room  near  the  JSEP  class)  or  with  several  staff  members  at  one  time.  The 
numbers  of  persons  interviewed  at  each  location  are  recorded  in  Table  4.1. 


Table  4.1.  Numbers  of  Persons  Interviewed  at  JSEP  Sites  (N=145) 


Site 

Current 

Enrollees 

Graduates/ 

OroDOuts 

SuDervi sors 

Education 

Staff 

MicroTICCIT: 

Fort  Lewis 

11 

7 

_  _ 

11 

Fort  Riley 

7 

6 

10 

8 

PLATO: 

Fort  Leonard  Wood 

6 

10 

_  _ 

8 

Fort  Sill 

17 

2 

8 

Fort  Bliss 

9 

5 

2 

2 

Fort  Jackson 

5 

1 

1 

9 

TOTALS 


55 


29 


15 
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Interviewers  used  standard  questions  ("JSEP  Interview  Protocols")  as  probes  to  start 
the  conversations,  and  these  are  included  in  Appendix  B.  The  questions  were  quite 
general,  such  as,  "What  do  you  want  to  get  out  of  JSEP?  How  does  it  compare  with 
other  classes  vou’ve  taken?"  Much  of  this  content  was  also  addressed  in  the  attitude 
surveys  prepared  for  the  evaluation,  so  we  did  not  attempt  to  collect  the  same  information 
exhaustively  from  every  interviewee.  Instead,  we  tried  to  get  JSEP  participants  to  tell  us 
what  they  wanted  to  say  about  the  program. 

The  observations  were  equally  unstructured.  Between  interviews  we  watched  soldiers 
using  JSEP,  occasionally  asking  them  or  their  instructor  to  explain  an  activity,  and  noting 
how  the  soldiers  worked.  The  focus  here  was  on  describing  JSEP’s  use. 

Soldier  Attitude  Survey 

The  JSEP  instructors  were  asked  to  have  each  soldier  complete  the  Post-JSEP 
Attitude  Survey  (reproduced  Appendix  B)  upon  completing  the  program.  Completed 
questionnaires  were  returned  to  AIR  for  175  soldiers  across  the  seven  pilot  sites.  The 
results  of  this  survey  are  given  in  Table  D.l,  Appendix  D,  which  also  includes  the 
verbatim  responses  soldiers  wrote  to  open-ended  questions  about  their  opinions  toward 
JSEP  and  suggestions  for  improving  it.  This  chapter  will  discuss  the  survey  findings  as 
they  relate  to  observations  from  the  site  visits.  The  reader  is  referred  to  Appendix  D  for 
a  full  display  of  the  data. 


Factors  Affecting  Operations 

The  JSEP  classes  were  not  conducted  in  a  vacuum,  and  it  is  important  to  understand 
their  context  before  considering  the  participants’  assessments  and  recommendations  about 
the  program. 

Involvement  with  JSEP 

Two  of  the  installations,  Fort  Lewis  and  Fort  Riley,  are  part  of  the  U.S.  Army 
Forces  Command  (FORSCOM)  that  have  been  pilot-testing  JSEP  on  the  MicroTTCCIT 
system  and  have  worked  with  JSEP’s  development  since  September  and  July  1985, 
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respectively.  The  JSEP  instructors  at  these  sites  were  under  contract  to  Florida  State 
University. 

The  remaining  installations  visited  were  PLATO  sites  and  part  of  the  U.S.  Army 
Training  and  Doctrine  Command  (TRADOC).  Two  of  these,  Fort  Leonard  Wood  and  Fort 
Sill,  were  original  developmental  sites.  Fort  Leonard  Wood  began  working  with  JSEP  in 
July  1985,  followed  by  Fort  Sill  in  August  1986.  Their  JSEP  instructors  were  employed  by, 
and  reported  to,  Florida  State  University.  The  instructors  at  Fort  Jackson  and  Fort  Bliss, 
which  had  begun  working  with  PLATO  in  the  summer  of  1987,  were  under  local  contracts 
and  did  not  report  formally  to  Florida  State  University. 

The  organizational  and  historical  contexts  were  related  to  variations  in  judgments 
about  JSEP.  Educators,  for  example,  tended  to  explain  local  programs  in  light  of  their 
setting:  TRADOC  posts  with  an  education  and  training  mission  contrasted  with  FORSCOM 
posts  that  emphasized  field  responsibilities.  Comments  from  sites  that  had  a  long  history 
of  participation  in  JSEP  often  appeared  to  be  colored  by  past  difficulties  or  explained  in 
terms  of  the  changes  they  represented.  Staff  at  the  sites  using  MicroTICCIT  expressed  a 
belief  that  this  would  not  be  the  delivery  system  for  JSEP,  and  thus  had  different 
projections  about  the  program’s  future  than  did  educators  who  worked  with  JSEP  on 
PLATO. 

Conditions  Surrounding  the  Evaluation 

Several  conditions  during  the  1988  fiscal  year  were  particularly  salient  to  the 
evaluation.  These  included: 

•  Field  confusion  about  support  requirements  in  the  Florida  State  University  JSEP 
contract.  At  the  December  1987  meeting  there  was  disappointment  about  the 
reduced  number  of  computer  terminals  and  subscriptions  to  PLATO  ports  that 
Florida  State  University  would  support,  and  about  the  lack  of  expected 
instructor  training.  The  contractor  was  also  unable  to  support  as  many 
instructor  positions  as  had  been  expected. 

•  Delays  in  completion  of  JSEP.  Not  all  JSEP  lessons  had  been  corrected,  and 
JSEP  Coordinators  expressed  the  feeling  that  their  notes  about  errors  in  lessons 
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and  in  lesson  tests  were  ignored.  Some  aspects  of  the  soldier  management 
system  (such  as  default  assignment  of  Learning  Strategy  lessons)  were  not  yet 
in  place  and  there  was  no  diagnostic  test  to  exempt  soldiers  from  lessons  on 
topics  in  which  they  were  already  competent.  At  the  December  1987  meeting 
three  kinds  of  activities  were  proposed  for  the  spring:  a  formal  program 
evaluation,  activities  to  develop  a  diagnostic  test,  and  scheduling  various  sites 
to  host  demonstrations  of  JSEP.  We  observed  several  instances  in  which 
instructors  confused  evaluation  activities  with  others  that  were  part  of 
completing  JSEP’s  development,  typically  when  several  different  activities  called 
for  collecting  similar  information  from  soldiers. 

•  Change  in  use  of  JSEP.  At  the  time  the  evaluation  began,  the  field  sites  had 
not  been  using  JSEP  in  strict  accordance  with  recommended  procedures.  The 
JSEP  Coordinators  were  unsure  of  eligibility  criteria  and  had  been  generally 
enrolling  only  soldiers  with  a  fairly  high  (in  the  8th  grade  range  or  above) 
reading  ability.  The  program  was  generally  used  as  a  supplement  or  complement 
to  BSEP,  and  as  a  GT  test  preparation  course.  Instructors  were  free  to  create 
prescriptions  rather  than  adhere  to  those  already  written  for  the  various  MOSs, 
and  a  number  of  "GT  prescriptions"  were  in  use.  During  the  evaluation  the 
instructors  were  asked  to  follow  standard  JSEP  practices.  JSEP’s  management 
system  was  also  altered  at  that  time  to  administer  lesson  pretests  and  to 
automatically  assign  Learning  Strategy  lessons  when  default  criteria  were 
tripped.  Thus  educators,  and  sometimes  soldiers,  often  talked  about  multiple 
program  formats  when  they  referred  to  JSEP  in  interviews. 

•  Fund  reductions.  Budget  reductions  unconnected  to  JSEP  were  imposed  on  the 
Army  Continuing  Education  System  in  January  1988,  and  by  the  close  of  the 
evaluation  period  funding  had  decreased  by  about  25  percent.  Much  of  this 
reduction  was  reported  by  local  Education  Center  administrators  to  have  been 
absorbed  by  basic  skills  programs.  Concerns  and  projections  about  financial 
support  were  evident  in  educators’  predictions  for  the  future  of  JSEP. 
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The  JSEP  Classrooms 

This  section  briefly  describes  the  physical  settings  in  which  we  observed  JSEP.  Sites 
are  identified  here  to  help  the  reader  form  a  picture  of  the  different  classes.  Subsequent 
sections  will  not  identify  the  installation  from  which  a  comment  or  observation  was  made. 

MicroTICCIT  sites.  The  JSEP  laboratory  at  Fort  Lewis  was  a  large,  well  lighted 
room  on  the  first  floor  of  a  building  constructed  during  World  War  II.  There  were  sue 
JSEP  personal  computers  set  up  in  cubicles  at  one  end  of  the  room,  and  the  JSEP 
instructor  had  a  desk  near  these.6  A  teacher’s  desk  and  several  desks  and  small  tables 
for  students  were  located  at  the  other  end  of  the  room.  There  was  no  door  closing  the 
room  from  the  hall,  and  while  the  room  was  generally  quiet  we  thought  there  was  some 
disruption  when  visitors  came  in  to  ask  where  other  offices  were  located,  or  when 
meetings  were  held  in  the  room.  The  MicroTICCIT  computer  was  in  a  separate  small,  air- 
conditioned  room  adjacent  to  the  instructor’s  desk.  The  JSEP  instructor  circulated 
constantly  among  the  students  during  our  observation,  asking  if  they  needed  help  and 
providing  instruction  when  it  was  required.  Classes  were  held  in  both  the  morning  and 
afternoon  so  that  a  total  of  12  students  could  be  served  on  any  day. 

Fort  Riley’s  classroom  was  in  a  room  at  the  back  of  the  one-story  First  Brigade 
Learning  Center,  a  converted  mess  hall  in  the  "Custer  Hill"  (enlisted  section)  of  the  post. 
This  was  about  five  miles  from  the  main  Education  Center  building,  and  so  a  counselor 
assigned  to  the  Learning  Center  had  more  daily  contact  with  the  JSEP  instructor  than  did 
the  JSEP  coordinator.  There  were  15  MicroTICCIT  stations.  The  main  computer  was  in 
the  "kitchen"  adjacent  to  the  classroom,  and  the  instructor  did  much  of  his  paperwork  in 
that  room.  Soldiers  worked  in  fairly  open  carrels  arranged  around  the  walls  of  the  room. 
The  lighting  was  good.  There  was  some  traffic,  since  a  back  entrance  to  the  building 
opened  into  this  room,  but  passers-by  did  not  seem  to  distract  students.  Separate  classes 
were  scheduled  for  the  morning  and  afternoon.  On  Wednesday  morning  the  instructor  held 
team  training,  at  which  soldiers  (often  three  to  a  computer)  from  a  given  unit  worked  on 

6The  number  of  students  served  depended  upon  the  number  of  functioning  terminals. 
This  could  change  from  day  to  day.  Immediately  prior  to  the  evaluation  the  sites  reported 
these  numbers:  Walter  Reed  AMC,  10;  Fort  Lewis,  7;  Fort  Riley,  13;  Fort  Sill,  11;  Fort 
Leonard  Wood,  8;  Fort  Bliss,  8;  Fort  Jackson,  10. 


58 


a  selected  lesson.  The  instructor  was  on  his  feet  constantly,  and  appeared  to  encourage 
students  or  to  relate  their  lessons  to  Army  jobs  as  much  as  he  helped  them  with 
instructional  or  mechanical  problems. 

PLATO  sites.  At  Fort  Bliss  JSEP  classes  were  conducted  in  a  small  room  with  three 
closed  windows  on  the  first  floor  of  a  building  constructed  during  the  Second  World  War. 
The  building  was  marked,  "Disposal  1988."  The  flourescent  lighting  was  adequate.  There 
were  problems  in  regulating  the  temperature  during  our  visit,  with  the  room  alternately 
uncomfortably  hot  or  cold.  The  room  contained  seven  terminals,  six  of  which  were  used 
for  JSEP,  and  desks  for  the  JSEP  instructor  and  PLATO  lab  technician.  Space  was 
crowded  when  all  of  the  terminals  were  occupied,  especially  when  the  BSEP  teachers  came 
in  to  talk  with  the  JSEP  instructor  and  PLATO  lab  technician.  The  noise  level  from 
people  moving  in  and  out  was  fairly  high.  The  PLATO  lab  was  next  to  the  JSEP  room, 
and  upstairs  the  building  contained  BSEP  classrooms,  a  small  office,  and  a  students’ 
lounge.  Two  sets  of  classes  were  held  each  day.  The  instructor  frequently  walked  around 
the  room,  asking  students  if  they  needed  help  and  commenting  on  their  work. 

JSEP  at  Fort  Jackson  was  housed  in  a  modem  Learning  Center  built  for  continuing 
education  activities.  Rooms  were  bright,  carpeted,  filled  with  plants,  and  generally  very 
comfortable.  JSEP  was  located  in  a  PLATO  lab  on  the  ground  floor  in  a  room  that  was 
crowded  but  well-lit  and  fairly  quiet.  Classes  were  held  for  four  hours  in  the  morning 
and  in  the  evening.  There  were  15  terminals  available  for  JSEP.  These  were  spread 
around  the  walls  of  the  room  and  along  a  row  of  face-to-face  carrels  down  the  center.  A 
work  table  used  by  the  instructor  was  at  one  end  of  the  room.  She  could  sit  at  this 
table  and  observe  four  or  so  students,  or  could  move  from  terminal  to  terminal.  The 
instructor  was  on  her  feet  most  of  the  time  we  observed  her,  often  giving  instruction  or 
overriding  program  errors.  Fifteen  students  began  JSEP  the  first  day  of  the  visit  and 
they  kept  the  instructor  and  the  JSEP  coordinator  constantly  busy.  Students,  particularly 
those  using  JSEP  in  the  evenings,  helped  one  another  with  advice  about  misunderstood 
test  questions,  how  to  handle  mechanical  problems,  which  lessons  to  choose,  and  similar 
matters. 

The  Learning  Center  holding  JSEP  at  Fort  Leonard  Wood  was  a  pleasant,  one  story 
building  that  was  constructed  originally  as  a  post  exchange.  There  were  a  number  of 
classrooms  (many  of  them  empty,  we  were  told,  because  of  recent  budget  cuts)  around  a 
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central  core  of  offices.  The  six  JSEP  terminals  were  in  one  of  these,  a  room  about  12 
feet  square,  adequately  lighted  although  there  were  no  windows.  The  terminals  were 
arranged  along  two  walls  in  half-carrels  that  allowed  soldiers  to  observe  one  another  or 
talk  if  they  leaned  back  in  their  chairs.  The  instructor’s  desk  and  file  cabinet  were  at 
the  back  of  the  room,  facing  the  terminals.  She  usually  remained  at  this  desk,  unless 
asked  for  help,  in  what  she  explained  was  a  deliberate  effort  to  intervene  as  little  as 
possible  with  the  computerized  lessons.  The  room  was  very  quiet. 

Fort  Sill  housed  JSEP  and  the  PLATO  learning  lab  in  a  big  metal-roofed  building 
that  was  constructed  to  store  large  vehicles  or  earth  moving  equipment.  Previously,  each 
program  was  allocated  half  the  building.  Following  reductions  in  Florida  State 
University’s  contract,  the  terminals  on  the  JSEP  side  had  been  disconnected  and  packed 
for  shipping,  and  classes  were  held  in  the  PLATO  lab.  This  room  contained  12  terminals 
devoted  to  JSEP  arranged  in  a  maze  of  large  cubicles  separated  by  5-foot  tall  partitions. 
To  prevent  glare  on  the  screens  the  overhead  lights  had  been  turned  off  and  there  was  a 
small  desk  lamp  in  each  cubicle.  The  room  was  very  quiet.  There  were  several  small 
offices  at  one  end,  and  the  PLATO  lab  coordinator  and  JSEP  instructor  had  desks  in  the 
classroom  area.  The  instructor  did  not  sit  down  during  the  time  we  observed,  but  walked 
from  cubicle  to  cubicle  watching  students  to  see  if  they  were  working  or  needed  help. 

Survey  description  of  logistics.  Most  of  the  soldiers  who  completed  the  attitude 
survey  following  JSEP  reported  that  they  had  been  enrolled  for  four  to  six  weeks  (59%), 
for  four  hours  a  day  (79%),  five  days  a  week  (86%).  Two-thirds  (67%)  completed  their 
prescription  of  JSEP  lessons  by  the  time  they  left  the  program.  Almost  all  (93%)  listed 
English  as  their  native  language. 


Soldiers’  Comments 

We  talked  with  84  soldiers  across  the  six  installations:  55  who  were  enrolled  in 
JSEP  and  29  who  were  earlier  graduates  (25)  or  dropouts  (4)  from  the  program.  The 
discussion  of  their  comments  will  note  when  remarks  were  influenced  by  whether  a  soldier 
participated  before  or  during  the  evaluation  period.  The  numbers  of  soldiers  making  the 
same  comments  are  reported  in  this  section  because  they  give  a  sense  of  the  relative 
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frequency  of  different  opinions  and  beliefs.  When  percentages  are  given  they  are  based 
on  the  number  answering  a  question. 

What  Soldiers  Wanted  From  JSEP 

Seventy-six  of  the  soldiers  gave  their  reasons  for  enrolling  in  JSEP.  The 
overwhelming  response  was  to  improve  GT  test  scores  (84%).  Some  went  on  to  explain 
why  the  GT  was  important.  These  soldiers  needed  score  improvements  to  reenlist,  qualify 
for  a  new  MOS,  increase  quality  points  for  promotion,  or  meet  criteria  for  Warrant 
Officer  School.  Soldiers  were  next  most  likely  to  see  JSEP  as  a  source  of  self- 
improvement  (13%)  that  would  help  their  math  or  sharpen  their  basic  skills.  Smaller 
numbers  said  they  were  taking  JSEP  to  prepare  for  the  SQT  (Skill  Qualification  Test)  (5%) 
or  prepare  for  other  education,  such  as  college  (12%).  One  soldier  appeared  to  be  joking 
said  he  was  in  class  because  he  was  "hungry  for  power,"  and  saw  JSEP  as  a  way  to 
prepare  for  the  college  education  that  would  lead  to  it.  Drill  sergeants  not  working  in 
their  MOS  were  most  likely  to  want  SQT  preparation.  Motivation  for  GT  improvement 
was  about  evenly  split  between  the  desire  to  qualify  for  a  new  MOS  and  the  need  to  be 
eligible  for  reenlistment. 

Survey  description  of  goals.  Among  the  175  soldiers  completing  the  attitude  survey, 
47  percent  said  they  had  enrolled  in  JSEP  because  they  wanted  to;  18  percent  had  done  so 
at  the  recommendation  of  their  commander.  The  23  percent  who  wrote  in  an  answer  to 
this  question  said  they  enrolled  to  improve  their  GT  scores.  When  asked  in  a  subsequent 
question  to  check  the  primary  reason  for  enrollment,  86  percent  marked  "to  improve  GT," 
and  1 1  percent  did  not  respond  to  the  question.  Five  other  reasons  (e.g.,  pass  SQT)  were 
checked  by  one  person  each.  Qualifying  for  a  new  MOS  may  also  have  been  a  motivation; 
38  percent  of  the  soldiers  did  not  like  their  duty  MOS  and  wanted  to  change  it. 

Achieving  Their  Goals 

Sixty-two  of  the  soldiers  who  were  interviewed  commented  on  whether  they  had 
reached  the  goals  they  had  set  for  themselves  with  JSEP.  For  many  of  the  current 
enrollees  this  was  an  estimate  of  whether  they  would  reach  these  goals  by  the  time  they 
had  finished  the  program. 
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The  great  majority  (74%)  said  "yes,"  they  had  achieved  what  they  wanted  entirely  or 
to  some  degree.  Some  graduates  cited  increases  in  GT  or  TABE  scores  that  they 
attributed  to  JSEP;  one  soldier  said  that  he  had  passed  an  entrance  test  for  flight  school. 
Others  simply  said  that  they  had  learned  what  they  wanted,  or  that  they  thought  they 
would  reach  their  GT  score  objectives.  Some  of  the  answers  were  qualified.  One  soldier 
thought  JSEP  would  help  a  little  with  GT  testing,  but  not  at  all  with  his  MOS.  Another 
graduate  said  JSEP  had  no  effect  on  his  job,  but  put  him  ahead  of  others  in  his  BSEP 
class.  (The  practice  at  all  installations  was  to  enroll  soldiers  in  BSEP  II  or  some  other 
GT  preparatory  activity  after  they  had  completed  JSEP.) 

About  18  percent  of  those  who  responded  to  this  question  said  that  they  had  not 
reached  their  goals  with  JSEP.  One  dropout  noted  that  he  had  learned  a  little  about 
computers  but  that  was  not  what  he  had  enrolled  for.  Others  said  JSEP  was  not  job 
relevant.  The  remainder  (8%)  were  unsure  about  whether  they  met,  or  would  meet,  the 
goals  they  had  set  for  JSEP. 

Survey  reports  of  overall  effects.  Several  of  the  survey  questions  completed  by  the 
175  participants  dealt  with  global  effcts  of  JSEP.  Seventy-seven  percent  believed  that 
JSEP  had  increased  their  motivation  to  learn;  only  2  percent  thought  they  had  become  less 
motivated  due  to  the  program.  When  asked  how  much  they  had  learned,  54  percent  said 
"a  lot,"  and  42  percent  said,  "some."  In  general,  they  were  of  the  opinion  that  JSEP  had 
helped  the  most  by  improving  their  math  skills  (70%),  improving  their  reading  or  verbal 
skills  (37%),  or  teaching  them  how  to  use  a  computer  (32%). 

The  survey  group  was  more  positive  than  the  interviewees  about  JSEP’s  applicability 
to  job  performance.  Twenty-one  percent  believed  that  the  computer-based  instruction 
helped  them  learn  or  improve  job  skills.  When  asked  what  effect  JSEP  had  on  how  well 
they  did  their  jobs,  25  percent  thought  they  had  learned  much  of  use  and  42  percent 
thought  it  had  helped  a  little. 

Benefits  and  Problems 

The  question  of  goal  achievement  was  close  to  that  of  secondary  benefits  or 
problems  associated  with  JSEP  participation.  Twenty-six  of  the  interviewed  soldiers  (31%) 
did  not  suggest  any  benefits  gained  or  problems  encountered  through  JSEP.  Of  those  who 
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did  talk  about  this  area,  52  percent  said  that  the  lessons  had  not  been  relevant  to  their 
jobs.  Others  disagreed.  Some  thought  all  or  parts  of  the  lessons  were  useful  on  the  job 
(17%);  that  JSEP  had  helped  with  unit  training  (5%);  or  that  it  helped  prepare  for  the  SQT 
(7%).  One  soldier  believed  JSEP  had  enabled  him  to  qualify  for  reenlistment  while  another 
expressed  the  idea  of  prerequisite  competencies  when  he  told  how  JSEP  had  taught  him 
the  reading  and  math  skills  he  needed  to  learn  his  job.  About  22  percent  thought  that 
JSEP  lessons  were  relevant  to  the  Common  Tasks  Test  (CTT). 

Only  three  soldiers  mentioned  attitudinal  outcomes.  One  dropout  said  the  faulty 
computer  program  that  refused  to  accept  his  correct  answers  made  him  feel  stupid.  On 
the  other  hand,  two  soldiers  volunteered  that  JSEP  made  them  feel  good  about  themselves. 

Seventeen  soldiers  made  some  reference  to  the  effects  of  JSEP  in  pulling  them  away 
from  their  jobs.  Of  this  group,  47  percent  said  there  were  no  problems  -  they  had  good 
unit  support,  a  commander  who  actively  encouraged  education,  and  the  like.  Thirty-five 
percent  had  encountered  some  problems  -  missing  work,  or  the  feeling  that  there  was 
resentment  from  their  peers  or  supervisors.  A  few  (12%)  had  not  seen  resentment,  but 
had  been  assigned  night  duty  while  in  JSEP  and  were  simply  tired.  An  equivalent  number 
had  been  pulled  out  of  class  for  work  responsibilities  and  saw  this  disruption  as  causing 
problems  in  their  learning. 

Survey  descriptions  of  secondary  effects.  One  survey  question  asked  soldiers  to  note 
whether  participation  in  JSEP  had  resulted  in  any  peripheral  benefits  or  difficulties.  As 
with  the  interviews,  the  survey  data  uncovered  few  resultant  problems.  Thirty-one 
percent  said  there  were  no  major  problems.  A  few  (4%)  felt  their  commander  had  not 
liked  them  spending  time  in  JSEP  or  that  the  time  spent  in  class  had  held  them  back  from 
their  units.  The  survey  soldiers  were  more  likely  to  check  that  they  had  enjoyed  benefits 
from  JSEP:  motivation  to  learn  (57%),  increased  self-confidence  (45%),  a  break  from 
routine  (43%),  or  improved  GT  scores  (34%). 


JSEP  and  career  goals.  In  talking  with  soldiers  about  what  they  expected,  and  saw 
themselves  getting,  from  JSEP  it  became  clear  that  they  view  education  from  the 
perspective  of  their  careers.  Test  scores  determine  enlistment  eligibility,  promotion,  and 
access  to  desired  MOSs;  they  serve  as  gates  to  the  further  education  that  is  a 
determinant  of  the  same  benefits.  When  JSEP’s  contributions  to  MOS  performance  were 
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cited,  it  usually  appeared  to  be  an  afterthought:  "some  of  that  math  comes  up  on  my 
job."  In  general,  the  soldiers  interviewed  during  this  evaluation  were  in  JSEP  to  improve 
their  GT  scores;  thought  they  would  do  so;  and,  in  some  cases,  believed  that  this 
improvement  would  be  because  of  JSEP.  It  should  be  stressed  that  these  were  the 
soldiers’  perceptions.  Actual  performance  changes,  reported  in  Chapter  3,  were  small. 

Attitudes  Toward  JSEP 

The  preceding  comments  may  seem  like  faint  praise,  but  this  should  not  imply  that 
the  soldiers  did  not  value  JSEP.  They  did.  Some  91  percent  of  those  who  had  an  opinion 
about  JSEP  liked  the  program,  and  found  it  to  be  easy,  interesting,  challenging  and  fun. 
Many  soldiers  said  they  simply  enjoyed  having  the  lessons  on  computer.  One  student  said 
that  he  became  so  absorbed  in  lessons  that  he  lost  all  sense  of  time.  Others  said  they 
did  not  become  bored  and  fall  asleep  with  JSEP  as  they  had  with  human  teachers.  (At 
only  one  site  did  we  observe  soldiers  falling  asleep  during  afternoon  classes.) 

Attitudes  reported  in  surveys.  The  survey  asked  soldiers  to  note  whether  they  had 
liked  any  of  a  number  of  characteristics  of  JSEP.  The  largest  proportion  said  they  liked 
the  things  they  had  learned  in  the  lessons  (64%).  Fifty-three  percent  each  appreciated 
the  chance  to  leam  and  study  by  themselves  and  the  chance  to  use  a  computer.  They 
also  enjoyed  the  way  that  JSEP  explained  things  (43%). 

When  asked  what  they  disliked,  36  percent  of  the  survey  soldiers  responded,  "Nothing 
in  particular."  The  negative  factors  cited  most  often  were  problems  with  the  computer 
(26%),  errors  in  the  JSEP  lessons  (21%),  and  the  way  the  computer  responded  to  their 
answers  (15%).  About  half  of  the  soldiers  liked  JSEP  a  lot  when  they  began  (54%);  a 
larger  number  (69%)  liked  it  a  lot  when  they  finished. 

Surveys  and  interviews:  JSEP  and  learning  style.  JSEP  appealed  particularly  to  one 
kind  of  student.  This  was  the  soldier  who  had  a  low  tolerance  for  lectures  and  books, 
who  disliked  reading,  and  who  wanted  individualization  in  pacing  and  instruction.  During 
interviews,  these  learners  praised  JSEP  because  it  was  easier  than  classes  and  required 
less  reading  and  writing.  The  word  "boring"  appeared  often  in  their  comments  when  they 
talked  about  school  or  studying. 
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A  second  type  of  learner  saw  JSEP  as  an  efficient  and  painless  way  to  revive 
knowledge  forgotten  since  high  school.  As  one  put  it,  JSEP  was  a  review  of  "Things  you 
know  you  knew."  These  soldiers  often  referred  to  JSEP  as  a  "refresher"  and  were 
attracted  by  its  self-paced,  individualized  design  and  by  the  privacy  it  offered. 

A  smaller  proportion  of  soldiers  (9%)  did  not  like  JSEP.  When  they  gave  a  reason  it 
was  dislike  of  the  computer:  "Sitting  in  front  of  a  computer  four  hours  a  day  is  boring 

as - "  Or,  they  simply  preferred  to  be  taught,  as  one  man  put  it,  "Cave-man  style,  by 

an  old  fashioned  human  teacher." 

The  attitude  survey  asked  specifically  about  the  congruence  between  JSEP  and  the 
soldiers’  preferred  learning  styles.  These  soldiers  felt  JSEP  explained  concepts  clearly 
(56%),  showed  them  how  to  carry  out  skills  step-by-step  (74%),  gave  practical,  useful 
illustrations  (53%),  and  allowed  them  the  opportunity  to  practice  new  learning  (54%).  They 
felt  that  JSEP  helped  them  to  learn  at  their  own  pace  (87%),  practice  until  they  got  a 
task  right  (89%),  see  material  step-by-step  (89%), and  discover  how  to  do  things  on  their 
own  (87%).  When  asked  if  anything  about  JSEP  frustrated  them  or  prevented  them  from 
learning,  24  percent  said  they  didn’t  have  any  trouble  learning  with  the  program.  Those 
who  did  mention  instructional  frustrations  were  most  likely  to  cite  not  being  able  to  see 
what  they  got  wrong  on  tests  (63%),  or  that  the  illustrations  took  too  long  (31%).  (It 
should  be  noted  that  the  graphic  displays  were  slower  on  the  PLATO  system  than  on 
MicroTICCIT.)  In  responding  to  a  question  about  whether  they  learned  best  from  a 
teacher  or  a  computer,  54  percent  said  that  they  did  not  prefer  one  format  consistently. 

The  other  responses  were  balanced:  17  percent  preferred  the  computer,  15  percent 
preferred  a  teacher. 

About  one-fourth  of  the  interviewed  soldiers  volunteered  a  comparison  between  JSEP 
and  BSEP.  Of  that  group,  two  out  of  three  preferred  JSEP,  usually  saying  it  was  less 
boring  than  BSEP  or  that  it  let  the  learner  move  at  his  or  her  own  speed.  Five  soldiers 
thought  the  two  programs  were  basically  equivalent  and  liked  them  equally.  Three  said 
that  they  learned  more  in  BSEP  because  it  was  a  paper-and-pencil  course.  On  the  survey, 

52  percent  of  the  soldiers  believed  JSEP  should  be  used  as  a  part  of  BSEP.  Some  23 
percent  believed  the  computer-based  program  could  completely  replace  BSEP. 
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Perceived  Strengths  of  JSEP 


Not  unexpectedly,  many  of  the  characteristics  that  soldiers  identified  as  JSEP’s 
strong  points  were  the  ones  that  made  them  like  the  program.  The  computer  itself  was 
seen  as  a  good  teacher,  as  a  motivation  to  learn,  or  simply  as  less  boring  than  a  lecture 
and  workbook  format.  Soldiers  who  liked  the  computer  said  it  kept  their  interest  up  and 
made  time  fly.  Some  others  of  the  reported  strengths  were  basic  aspects  of  computer- 
based  instruction.  Soldiers  liked  working  at  their  own  pace;  not  having  to  wait  for  others 
as  they  might  in  a  lecture  class;  being  able  to  review  material;  and  feeling  that  the 
lessons  and  instruction  were  individualized.  They  appreciated  the  privacy  they  had,  and 
described  the  computer  as  an  "infinitely  patient"  teacher.  Soldiers  also  said  they  valued 
the  efficiency  of  the  system.  A  number  cited  the  pretests  (which  were  added  for  the 
evaluation  and  available  to  only  the  two-thirds  of  the  PLATO  system  interviewees  who  had 
not  completed  JSEP  before  then)  as  a  particularly  good  aspect.  Several  liked  the  idea  of 
short  lessons  that  gave  a  brief  review  followed  by  more  detailed  long  lesson  instruction 
only  if  it  were  needed.  The  flexible  scheduling  was  described  as  a  benefit.  There  was 
also  recognition  of  the  control  the  management  system  exerted.  Soldiers  said  JSEP  was 
effective  because  "You  can’t  cheat,"  or,  "It  won’t  let  you  go  on  until  you’ve  got  it  right." 
Several  said  that  the  strictness  of  the  test  formats,  which  could  reject  a  substantially 
correct  answer  because  it  was  not  stated  in  the  required  manner,  taught  self-discipline. 

This  was  echoed  in  remarks  about  the  content  Soldiers  thought  one  of  the 
strengths  of  JSEP  was  the  way  in  which  it  presented  material  step-by-step  and  in  more 
detail  than  a  textbook  or  teacher  might.  They  believed  the  directions  were  clear  and 
comprehensive,  and  the  content  was  understandable.  A  few  soldiers  mentioned  the  review 
of  materials  when  a  test  was  failed  as  a  good  thing.  Others  cited  the  graphics  as  one  of 
the  best  parts  of  JSEP,  while  some  picked  out  the  Introduction  and  Time  Management 
Lesson  as  particularly  helpful. 

On  the  other  hand  were  those  who  thought  JSEP  worked  because  of  what  it  didn’t 
require.  They  stated  they  liked  JSEP  because  there  was  less  reading  and  writing,  and  not 
as  much  pressure  to  produce,  as  in  other  learning  settings.  A  few  thought  its  strongest 
point  was  its  games.  (There  are  games  included  in  some  of  the  JSEP  lessons.) 
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Math  was  mentioned  as  a  strength  more  often  than  language  skills  were,  especially 
when  soldiers  described  JSEP  as  a  "refresher."  JSEP  was  considered  a  good  review  of 
fractions  and  geometry.  A  few  soldiers  thought  it  was  strong  in  developing  vocabulary 
and  reading  comprehension.  Comments  about  the  content  overall  described  it  as  a  good 
review  of  material  forgotten  since  high  school  or  Army  common  task  training,  and  as 
preparation  for  tests  such  as  the  CTT,  GT,  or  GED  (General  Educational  Development) 
test. 


The  JSEP  instructor  was  frequently  mentioned  as  a  strength  of  the  program.  He  or 
she  was  helpful,  supportive,  and  able  to  explain  material  when  the  computer  instruction 
did  not  suffice,  in  the  soldiers’  opinions.  At  times  it  sounded  as  if  the  soldiers  were 
describing  a  team-teaching  relationship  between  JSEP  and  the  human  instructor. 

Perceived  Weaknesses  in  JSEP 

Sixteen,  or  about  one  in  five  of  those  interviewed,  either  failed  to  mention  any 
weaknesses  in  JSEP  or  said  explicitly  that  it  had  none.  Among  soldiers  who  did  report 
problems,  two  came  up  with  notable  frequency. 

The  first  was  that  of  errors  or  difficulties  with  the  content  or  management  system. 
Soldiers  reported  that  test  answers  were  wrong,  that  touch  screens  refused  to  accept  their 
responses,  that  there  were  errors  in  the  content  of  the  lessons.  It  should  be  noted  that 
not  all  of  what  soldiers  saw  as  errors  might  be  viewed  as  such  by  program  developers. 

From  some  remarks  it  appeared  that  learners  saw  the  management  system’s  accepting  "2 
1/4  Gal"  as  an  answer  but  not  "9/4  Gal"  (this  example  is  fictitious)  as  an  error  when  they 
were  not  specifically  told  to  reduce  their  answer  to  lowest  terms. 

Tests  were  the  second  major  source  of  perceived  weaknesses.  Many  soldiers  thought 
JSEP  would  be  improved  if  they  were  told  which  items  they  missed  on  a  lesson  test;  many 
of  these  wanted  the  review  lesson  that  followed  a  failed  test  to  be  limited  to  the  area 
failed,  not  to  include  an  entire  lesson.  Not  knowing  what  they  had  answered  wrong,  and 
being  made  to  repeat  an  entire  lesson,  aroused  some  hostility;  one  enlisted  woman  said, 
"Sometimes,  if  that  machine  had  a  neck _ " 
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A  few  soldiers  did  not  like  the  lesson  pretesting  that  was  instituted  for  the 
evaluation.  Most  of  these  simply  wanted  the  option  of  taking  a  lesson  if  they  disagreed 
with  the  pretest’s  decision  that  they  already  knew  the  content.  A  few  said  they  knew 
from  a  lesson’s  title  whether  or  not  they  needed  to  take  it,  and  that  they  felt  the  pretest 
was  embarrassing  or  a  waste  of  time. 

Some  soldiers  did  not  like  computer-based  instruction.  Unlike  others,  they  found 
learning  from  a  computer  to  be  boring.  For  these,  JSEP  was  too  slow,  the  screen  gave 
them  a  headache,  they  wanted  human  interaction  with  a  teacher  ("You  don’t  have  to  push 
a  button  to  make  her  work!"),  or  they  simply  could  not  tolerate  three  or  four  hours  of 
concentrating  on  a  computer  terminal.  A  few  soldiers  thought  the  content  of  JSEP  was 
too  basic  for  their  needs.  More  found  the  requirement  to  repeat  an  entire  failed  lesson, 
coupled  with  the  slowness  of  the  PLATO  graphic  displays,  very  frustrating.  One  soldier 
calculated  that  he  spent  two  hours  a  week  waiting  for  an  instructor  to  help  him  log  on 
and  off  as  he  worked  on  paper-based  lessons,  and  cited  this  as  a  waste  of  time. 

There  were  soldiers  who  thought  the  paper  lessons  were  not  as  well  designed  as  the 
on-screen  ones,  or  that  moving  to  a  paper  lesson  broke  the  pace  and  concentration  they 
had  established  with  the  computer.  Several  thought  the  Introductory  and  Time 
Management  lessons  were  too  long  or  had  little  practical  use.  Other  soldiers  wanted 
additions  to  JSEP,  particularly  in  language  arts  areas.  They  wanted  lessons  expanded  or 
added  in  vocabulary  development,  paragraph  comprehension,  writing,  spelling,  or 
mathematics  word  problems.  A  few  said  the  absence  of  timed  tests  limited  JSEP’s  value 
as  preparation  for  the  GT  or  SQT. 

Soldiers  occasionally  thought  the  mathematics  lessons  did  not  explain  their  content 
adequately  or  that  the  rigidity  of  the  lessons  was  a  problem  -  if  a  student  did  not  learn 
it  the  first  time,  repeating  the  instruction  verbatim  probably  would  not  help,  in  their 
opinion.  One  man  complained  because  some  competencies  did  not  have  a  long  lesson;  in 
his  opinion,  if  you  failed  the  short  lesson  you  were  lost. 

In  summary,  a  relatively  small  number  of  soldiers  cited  as  weaknesses  basic  aspects 
of  computerized  instruction  that  cannot  be  changed.  These  learners  found  the 
organization  and  delivery  of  JSEP  incompatible  with  their  learning  styles.  Others  reported 
weaknesses  that  appear  simple  to  amend.  Problems  with  test  items,  getting  the  touch 
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screen  to  respond,  and  errors  in  the  lessons  themselves  should  be  readily  correctible.  In 
fact,  these  errors  are  not  listed  in  detail  in  this  report  because  JSEP’s  lesson  review 
process  has  been  collecting  this  information  for  several  years. 

Instructional  Critique:  Survey  Responses 

The  attitude  survey  asked  soldiers  to  give  their  subjective  ratings  of  a  number  of 
the  curricular  and  instructional  aspects  of  JSEP.  These  agreed  with,  and  expanded  upon, 
the  comments  that  have  been  discussed  as  perceived  strengths  and  weaknesses  of  the 
program. 

The  majority  of  the  soldiers  rated  all  of  the  major  program  elements  as  useful. 
They  described  as  "very  useful"  the  lesson  pretest  (63%),  the  short  lesson  (67%),  the  test 
after  the  short  lesson  (72%),  the  long  lesson  (55%),  and  the  test  after  the  long  lesson 
(62%).  The  soldiers  also  judged  the  Learning  Strategy  lessons  to  be  helpful,  particularly 
the  modules  on  test-taking  (89%)  and  problem-solving  (83%).  However,  soldiers  rated 
Learning  Strategy  lessons  as  useful  even  when  thay  had  not  taken  them,  which  led  us  to 
suspect  that  many  soldiers  evaluated  JSEP’s  overall  effect  on  test-taking  or  reading 
comprehension  skills  rather  than  the  individual  lessons  in  which  these  skills  were 
addressed  separately.  On  the  topic  of  lesson  assignment,  most  soldiers  checked  that  the 
lessons  they  had  been  given  were  the  ones  they  needed  (53%)  while  a  smaller  number 
(24%)  felt  the  assigned  lessons  were  mostly  not  the  ones  they  needed  or  wanted.  About 
one-third  (37%)  said  they  would  like  to  change  the  way  JSEP  is  set  up. 

Soldiers  generally  rated  specific  elements  of  JSEP  as  "excellent"  or  "good."  Only  a 
half  dozen  items  were  judged  to  be  "poor"  or  to  "need  improvement"  by  as  many  as  15 
percent  of  the  respondents.  These  were  the  computer’s  response  to  student  answers  on 
practice  questions  (19%),  the  way  tests  were  scored  (18%),  the  choice  of  lessons  assigned 
(21%),  the  length  of  lessons  (15%),  the  amount  of  time  spent  in  JSEP  (15%),  and  the 
civilian  content  of  the  lessons  (15%). 

Finally,  the  soldiers  were  asked  if  they  had  experienced  any  of  several  problems  in 
using  JSEP.  The  survey  respondents  did  not  judge  the  program  in  general  to  be  difficult. 
Most  felt  the  lessons  were  at  the  correct  level  of  difficulty  (61%);  a  few  (3%)  throught 
they  were  too  easy  but  no  one  thought  they  were  too  hard.  When  asked  what  was  most 
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difficult  about  JSEP,  46  percent  responded  that  nothing  was  difficult.  Some  (39%) 
reported  problems  in  maintaining  their  concentration  while  they  worked  with  the  program. 

About  one-third  (31%)  reported  having  no  computer  problems.  Others  said  they  had 
encountered  difficulties  when  the  JSEP  program  marked  a  correct  answer  wrong  (42%),  got 
"stuck"  in  a  lesson  (e.g.,  the  screen  froze  or  program  errors  would  not  let  a  lesson 
continue  --  2\>  percent),  or  the  computer  did  not  respond  to  their  commands  (18%). 

Advice  and  Recommendations  About  JSEP 

Even  though  the  soldiers  were  able  to  point  out  weaknesses  in  JSEP,  their  usual 
recommendation  was  that  the  Army  keep  the  program  and  many  volunteered  that  they 
would,  "Recommend  JSEP  to  anyone."  Several  interviewees  and  61  percent  of  the  survey 
respondents  suggested  that  no  changes  be  made.  Others  thought  that  JSEP  should  develop 
language  arts  lessons  to  provide  more  comprehensive  preparation  for  career  tests  or 
further  formal  education.  They  recommended  that  JSEP  add  prescriptions  so  that  it  would 
cover  all  of  the  MOSs  at  any  installation  using  it,  and  some  suggested  that  MOS  training 
could  be  logically  added  to  JSEP’s  pre-MOS  education.  There  were  those  who  thought 
that  college  preparatory  courses  and  college  credit  courses  could  be  taught  in  a  computer- 
based  format  similar  to  JSEP’s. 

The  students  made  recommendations  about  the  audiences  and  uses  for  JSEP.  Several 
soldiers  advised  that  it  be  used  to  teach  common  tasks  to  recruits  before  their  duty 
assignments,  or  that  all  soldiers  with  a  GT  of  less  than  110  be  referred  to  the  program. 
Others  saw  JSEP  as  a  general  basic  skills  refresher  that  they  would  suggest  for  anyone 
who  had  been  out  of  high  school  for  a  while. 

While  a  few  soldiers  said  that  JSEP  could  replace  BSEP,  a  larger  number  advised  the 
Army  to  keep  both  programs.  This  agreed  with  the  52  percent  of  the  survey  respondents 
who  believed  JSEP  should  be  used  as  a  part  of  BSEP.  JSEP  and  BSEP  were  either  seen  bv 
those  with  whom  we  talked  as  complementary  to  one  another  or  as  alternatives  for 
soldiers  who  have  different  learning  styles.  A  number  warned  that  JSEP  should  not  drop 
its  instructors.  They  thought  the  program  would  lose  effectiveness  without  its  human 
partner.  This  was  somewhat  in  disagreement  with  the  survey  responses.  In  those  data,  33 
percent  of  the  soldiers  noted  that  they  "rarely  or  never"  went  to  the  instructor  for  help. 
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When  assistance  was  requested,  it  was  most  likely  to  be  with  problems  in  understanding 
lessons  (38%)  or  with  computer  problems  (31%).  The  soldiers  said  the  instructors  helped 
by  offering  an  alternative  explanation  (55%)  or  adjusting  the  computer  (31%).  One 
possible  explanation  for  the  discrepancy  between  interview  and  survey  data  is  that  soldiers 
rarely  asked  for  help  because  the  instructors  generally  spent  their  time  circulating  among 
the  students,  providing  assistance  before  the  soldiers  requested  it.  Finally,  several 
soldiers  pointed  out  that  learning  from  JSEP  takes  concentration.  Completing  a 
prescription  on  their  own  once  the  allocated  time  had  run  out,  or  working  lessons  after 
being  on  duty  all  night,  was  difficult.  They  recommended  keeping  JSEP  as  an  on-duty 
program  and  setting  up  procedures,  such  as  block  pacing,  that  would  minimize  the  chances 
of  a  learner’s  being  pulled  for  duty  in  the  middle  of  a  prescription. 

Education  and  Soldiers 

We  talked  with  soldiers  about  JSEP  because  we  believe  that  how  a  program  is 
perceived  will  affect  how  it  is  used,  and  because  it  seemed  logical  that  a  program 
customized  to  teach  MOS  prerequisite  competencies  would  also  be  customized  for  its  Army 
users.  A  major  theme  that  emerged  from  these  interviews  was  that  the  JSEP  participants 
were  a  distinctive  type  of  learner.  Admittedly,  a  small  number  gave  the  impression  they 
had  enrolled  in  JSEP  because  it  was  easier  than  their  jobs,  and  a  few  didn’t  like  the 
Army.  But  the  great  majority  reflected  some  common  beliefs  and  values  that  distinguished 
them  as  Army  learners: 

•  Education  (in  this  instance,  JSEP)  is  valuable  as  it  contributes  to  a  mission. 
JSEP  is  a  basic  skills  program.  The  perceived  mission  of  basic  skills  programs 
is  to  improve  performance  on  career  tests:  GT  or  SQT.  Therefore,  JSEP  is 
seen  as  good  to  the  extent  that  it  improves  test  performance. 

•  Efficiency  appeared  to  be  highly  valued.  Errors  in  instructional  content,  lesson 
tests,  or  the  programming  that  governed  the  manner  in  which  JSEP  responded 
to  a  learner  were  very  poorly  tolerated.  The  soldiers  using  JSEP  appeared  to 
be  less  forgiving  of  flaws  in  the  program  than  we  would  have  expected  a 
comparable  civilian  audience  to  be. 
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•  We  perceived  time  to  be  the  enlisted  member’s  major  currency  in  the  social 
economy  of  the  Army.  Any  sense  that  an  educational  program  is  wasting  time 
gives  the  message  that  the  soldier’s  worth  is  being  discounted,  and  produces  an 
emotional  reaction  that  may  surprise  the  more  leisurely-oriented  civilian 
observer. 

•  Soldiers  use  information  (education,  training,  manuals)  to  do  their  jobs.  While 
there  was  great  faith  in  the  instrumental  value  of  education  as  it  contributed  to 
personal  development,  there  was  very  little  awe.  To  many  soldiers,  witholding 
information  about  which  test  items  have  been  missed,  or  using  a  pretest  to 
determine  whether  they  can  take  a  lesson,  is  like  presenting  them  with  a 
defective  tool.  They  do  not  seem  prepared  to  accept  JSEP’s  design  on  faith, 
but  voiced  instead  a  "show  me"  attitude. 

In  developing  JSEP,  great  care  was  taken  to  select  competencies  that  related  to 
Army  job  duties,  and  to  use  military  situations  and  language  wherever  possible  in  the 
lessons.  We  suggest  that  equal  attention  be  paid  to  "greening"  the  operational  aspects  of 
JSEP  to  reflect  Army  values  and  beliefs.  This  should  include  great  attention  to  making 
JSEP  as  error-free  and  efficient  as  possible  and  to  explaining  concisely  and  convincingly 
how  it  contributes  to  the  Army's  and  the  individual  soldier’s  mission. 


Supervisors’  Comments 

The  JSEP  coordinator  at  each  site  was  asked  to  arrange  interviews  with  a  few 
training  or  duty  supervisors  of  soldiers  who  had  completed  JSEP.  At  two  sites  no 
interviews  were  scheduled  because  the  coordinators  felt  that  the  soldiers’  supervisors 
would  not  know  much  about  JSEP  or  because  of  time  constraints.  At  one  site,  on  the 
other  hand,  we  talked  with  10  supervisors  of  soldiers  who  had  participated  in  JSEP  -- 
two-thirds  of  our  total  15  interviews.  Thus  the  distribution  of  supervisors  is  uneven 
across  the  installations  piloting  JSEP.  Furthermore,  because  of  timing,  these  were 
supervisors  uf  soldiers  who  had  completed  JSEP  before  the  evaluation  began. 

The  supervisors  (sergeants)  with  whom  we  talked  represent  :d  supply  units,  motor 
pools,  ci'Oks,  and  clerical  workers  as  well  as  other  areas.  Several  had  more  than  20  years 


72 


of  experience  in  the  Army.  There  was  considerable  departure  from  the  prepared  interview 
questions  with  this  group,  because  it  appeared  that,  while  the  supervisors  rarely  had 
detailed  knowledge  of  JSEP,  they  did  see  themselves  playing  a  crucial  role  in  soldiers’ 
continuing  education.  They  generally  viewed  JSEP  as  a  new  form  of  BSEP  and  did  not 
distinguish  the  programs  in  their  conversation. 

Knowledge  of  JSEP 

At  the  site  providing  the  largest  numbers  of  interviews  the  supervisors  were 
relatively  knowledgeable  about  JSEP.  Many  mentioned  having  visited  the  lab  and  talked 
with  the  instructor.  They  said  the  computerized  instruction  was  "amazing,"  and  that 
observing  JSEP  allayed  their  doubts  about  whether  it  could  work.  One  sergeant  said  that 
JSEP  did  not  have  the  "for  dummies"  (his  words)  image  of  BSEP,  while  another  liked  the 
idea  of  computers  even  if  JSEP  took  more  time  than  BSEP  had.  Another  also  believed 
JSEP  was  less  efficient  than  BSEP  sirce  it  did  not  diagnose  and  teach  only  the  soldier’s 
weak  areas,  but  thought  JSEP  was  still  good  preparation  for  the  SQT.  There  was  approval 
of  the  self-paced  aspect  of  instruction,  with  the  warning  that  the  soldier  must  be  willing 
to  learn  for  it  to  work.  Some  of  the  sergeants  were  familiar  with  Army  education 
because  they  themselves  had  taken  BSEP  classes.  Those  who  said  they  had  little 
knowledge  of  JSEP  relied  on  counselors  to  place  soldiers  in  programs. 

Those  who  were  familiar  with  JSEP  saw  it  as  either  BSEP  on  a  computer  or  as  a  GT 
improvement  program.  Several  thought  it  would  also  help  prepare  soldiers  for  the  SQT, 
GED  Test,  or  college  admissions  Scholastic  Aptitude  Test  (SAT). 

Supervisor’s  Role  in  Conti  uing  Education 

At  or  i  installation  the  supervisors  described  having  a  limited  role  in  their  soldiers’ 
education.  These  sergeants  received  a  list  of  soldiers  with  low  GT  scores  from  the 
Education  Center,  and  then  signed  off  on  the  counselor’s  recommendation  for  soldiers 
referred  to  classes.  One  other  supervisor  described  a  similarly  uninvolved  role,  saying 
that  he  "Doesn’t  stop  people  from  going  to  school,  it’s  good  for  the  soldier  and  good  for 
the  Army,"  but  not  describing  anything  he  did  to  promote  participation.  And  there  was  a 
single  supervisor  who  talked  about  screening  soldiers  who  wanted  continuing  education,  to 
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be  sure  that  they  had  good  reenlistment  potential  and  were  not  after  "sham  status"  that 
would  relieve  them  from  onerous  duty. 

The  remaining  nine  supervisors  talked  about  actively  promoting  participation  in  BSEP 
or  JSEP  among  the  soldiers  in  their  charge.  One  said  that,  "Promoting  education  is  just 
looking  after  your  troops,"  while  another  said  that,  "The  more  you  help  your  soldiers  the 
more  it  helps  you."  They  believed  that  supervising  was  synonymous  with  training,  and 
that  a  large  part  of  their  job  was  to  improve  soldiers  and  help  them  better  themselves. 

The  sergeants  did  this  by  reviewing  the  records  of  soldiers  newly  assigned  to  their  units 
to  see  which  ones  needed  to  gain  a  GED  or  raise  their  GT  scores.  They  then  referred 
eligible  soldiers  to  the  Education  Center,  sometimes  taking  the  soldier  there  to  meet  with 
a  counselor  or,  for  a  particularly  good  soldier,  calling  the  Education  Center  to  plan  how 
the  soldier’s  work  would  be  scheduled  around  classes.  A  few  sergeants  said  they  checked 
with  the  Education  Center  periodically  to  monitor  a  soldier’s  progress. 

The  supervisors  gave  two  rationales  for  this  kind  of  careful  attention.  The  first  was 
pragmatic:  developing  good  soldiers  was  their  job  and  what  they  were  evaluated  on.  The 
second  was  a  form  of  mentoring.  Several  sergeants  said  they  encouraged  basic  skills 
education  because  BSEP  had  been  instrumental  in  their  own  success  and  they  wanted 
others  to  have  this  opportunity.  One  sergeant  pointed  this  out  when  he  told  us  to  look 
at  a  supervisor’s  rank  and  age.  He  said  that  the  higher  an  NCO  has  gotten,  and  the 
longer  he  or  she  has  been  in  the  Army,  the  higher  this  NCO  would  rate  education. 

Effects  of  JSEP 


Although  several  supervisors  said  they  had  not  noticed  any  effects  yet  in  the  soldiers 
they  had  referred  to  JSEP,  others  felt  that  they  could  point  to  outcomes.  These  included 
mention  of  soldiers  having  raised  their  GT  scores  by  15  to  20  points.  One  said  the 
average  GT  score  in  his  unit  had  increased  by  23  points.  Another  sergeant  thought  his 
soldier  was  doing  better  on  the  job,  and  two  said  that  the  soldiers  they  had  referred  were 
now  more  reflective  —  they  thought  before  they  acted. 

Attitude  improvement  was  the  effect  cited  next  most  frequently  after  GT 
improvement.  Two  sergeants  made  general  comments  that  sending  soldiers  to  JSEP  or 
BSEP  made  the  enlisted  members  feel  that  somebody  cared  about  them,  or  that  "This  guy’s 
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helping  me,  so  I’ll  do  a  good  job  for  him."  Less  optimistic  was  the  sergeant  who  said 
that  all  but  one  of  the  soldiers  he  had  sent  to  BSEP  had  left  the  Army.  This  supervisor 
thought  continuing  education  was  too  often  abused  as  a  way  to  get  out  of  work. 

Role  of  JSEP  in  the  Future 

The  supervisors  talked  about  how  they  had  seen  continuing  education  evolve  during 
their  time  in  the  Army,  and  whether  they  thought  it  would  be  needed  in  the  near  future. 

They  believed  that  education’s  role  has  changed  in  the  Army.  For  example,  one 
sergeant  stated  that  when  he  enlisted,  education  was  what  you  did  with  "Guys  you  wanted 
to  get  out  of  the  way."  Now,  the  entire  Army  is  geared  to  run  on  education  and  test 
scores.  Today’s  recruits  were  described  as  better  educated  and  smarter,  more 
sophisticated,  than  those  of  10  years  ago.  When  one  sergeant  enlisted  22  years  ago,  "80 
percent  needed  BSEP.  Now  it’s  20  percent."  The  sergeants  said  that  many  Army  jobs  are 
more  complicated  than  they  were  in  the  past.  One  sergeant  thought  that  this  was  causing 
a  problem  in  training.  Trainers  today  taught  subjects,  not  people,  and  he  saw  young 
officers  unable  to  teach  enlisted  personnel  effectively  -  overwhelming  them  with 
technical  information  they  did  not  understand  and  leaving  them  "With  their  brains 
smoking." 

The  lone  dissenter  was  the  sergeant  who  thought  soldiers  often  took  advantage  of 
basic  skills  education.  He  believed  that  it  should  entail  pay-back  obligations  to  discourage 
soldiers  who  had  no  intention  of  reenlisting  from  using  JSEP  as  a  way  to  avoid  work  or 
to  get  a  free  GED. 

The  sergeants  saw  themselves  continuing  to  send  soldiers  who  had  good  attitudes  and 
job  performance  to  basic  skills  education  in  order  to  keep  them  in  the  Army.  They 
voiced  the  belief  that  education  makes  a  better  soldier,  and  that  the  opportunity  for 
education  (toward  which  BSEP  is  often  the  first  step)  is  a  major  reason  for  many  to 
enlist.  One  sergeant  remarked  that  if  funding  was  tight,  "There  are  other  things  to  cut 
besides  education."  Another  estimated  that  even  with  today’s  higher  education  standards 
for  enlistment,  about  one  in  five  recruits  had  some  problems  reading  Army  materials. 

And,  several  interviewees  said  both  Army  recruiters  and  immature  recruits  may  not  take 


75 


the  test  scores  as  seriously  as  they  should.  As  long  as  the  GT  score  plays  a  role  in 
quality  points  and  reenlistment  there  will  be  a  place  for  BSEP  or  JSEP. 

Summary 

These  15  sergeants,  selected  on  the  basis  of  their  having  encouraged  or  allowed  their 
soldiers  to  participate  in  JSEP,  do  not  form  a  generalizable  sample  of  Army  opinions 
toward  continuing  education.  They  do,  however,  give  one  picture  of  how  JSEP  can 
function  successfully  when  supervisors  view  educational  development  as  one  aspect  of  their 
role  in  managing  soldiers. 

These  superviors  generally  saw  JSEP  as  an  effective  means  for  improving  soldiers’  GT 
scores.  The  test  scores  have  a  direct  effect  on  career  outcomes  such  as  reenlistiment 
eligibility.  More  indirectly,  they  affect  attitude  and  job  skills  by  giving  soldiers  a  feeling 
of  success  and  opening  the  door  to  further  education. 

Developing  soldiers  is  a  large  part  of  the  supervisor’s  job,  in  the  opinion  of  those 
we  interviewed,  and  one  that  most  of  them  found  personally  rewarding.  Education  may  be 
a  logistical  problem  to  integrate  with  military  duties,  but  these  sergeants  did  not  see  it  as 
antagonistic  or  irrelevant  to  their  mission. 


Educational  Staff  Members’  Comments 

We  spoke  with  46  members  of  the  educational  staffs  at  the  pilot  sites.  These 
interviews  included  JSEP  instructors,  JSEP  coordinators,  counselors,  BSEP  teachers,  PLATO 
laboratory  coordinators,  and  local  Education  Center  administrators.  Because  we  believed 
that  JSEP  might  be  managed  differently  at  the  six  sites  we  asked  the  JSEP  coordinator  at 
each  to  direct  us  to  everyone  in  the  Education  Center  who  had  anything  to  do  with  the 
program.  WTiile  we  did  use  the  interview  questions  included  in  Appendix  B,  the  remarks 
usually  fell  into  three  categories:  the  respondent’s  evaluation  of  JSEP,  perception  of  the 
functions  it  fulfilled,  and  projections  about  its  future  use.  And,  paper-and-pencil  surveys 
were  completed  by  11  full-  or  part-time  JSEP  instructors.  These  survey  responses,  and 
the  additional  comments  that  the  instructors  wrote,  are  included  in  Appendix  E.  Chapter 
4  draws  upon  that  survey  data  where  appropriate. 
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Evaluations  of  JSEP 


The  majority  of  those  with  whom  we  talked  thought  JSEP  was  a  good  program  but 
were  also  able  to  point  out  its  shortcomings.  Generally  the  approval  was  global  ("This  is 
the  way  to  go")  while  the  criticisms  were  very  specific.  These  criticisms  are  detailed  in 
Appendix  E. 

Overall  interview  evaluations  ranged  from  the  belief  of  one  instructor  that  JSEP 
could  replace  all  of  the  existing  GT  and  SQT  improvement  programs  and  raise  any  soldier’s 
performance  on  those  tests  above  reenlistment  criteria,  to  the  opinion  of  another  that 
JSEP  was  inefficient  and  expensive  but  could  probably  be  useful  as  a  supplement  to  BSEP. 
Before  the  evaluation  most  installations  had  used  JSEP  as  a  supplement  to  BSEP,  and  when 
educators  volunteered  comparisons  of  the  two  programs  it  was  usually  to  say  that  JSEP 
could  not  replace  BSEP. 

Education  Center  staff  members  described  JSEP  as  user-friendly  and  highly 
interesting.  Working  with  the  computer  was  a  challenge  and  a  motivator.  Several  said 
that  they  had  never  seen  a  soldier  asleep  at  a  JSEP  terminal  but  couldn’t  say  the  same 
for  BSEP  classes.  They  opined  that  warrant  officers  and  other  senior  NCOs  who  would 
not  enroll  in  BSEP  because  they  would  feel  uncomfortable  showing  academic  deficiencies 
would  use  JSEP  because  the  instruction  was  individualized  and  all  mistakes  were  made  in 
private.  Instructionally,  the  concepts  of  long  and  short  lessons,  self-pacing,  and 
individualized  instruction  were  presented  as  strengths  in  JSEP.  The  graphics  and  games 
were  praised  for  their  interest  value.  One  teacher  thought  the  most  useful  aspect  of 
JSEP  was  that  it  taught  soldiers  to  follow  directions.  Logistically,  the  open-entry  open- 
exit  nature  of  JSEP  was  seen  as  very  efficient,  and  an  instructor  noted  that  the  expected 
management  system  would  be  a  useful,  time-saving  way  to  maintain  soldiers’  records. 

While  JSEP  was  consistently  described  as  a  good  mathematics  program,  it  was  just  as 
frequently  cited  for  not  addressing  reading  comprehension  skills.  Reading  comprehension 
is  important  for  all  jobs  because,  as  one  respondent  put  it,  "Reading  the  technical  manual 
is  the  first  prerequisite  competency  for  any  MOS."  The  only  way  in  which  JSEP  was  seen 
to  affect  reading  was  through  the  fact  that  all  of  the  instruction  required  reading—  either 
paper  materials  or  the  computer  screen.  This  could  be  something  of  a  problem,  and  JSEP 
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was  described  as  inappropriate  for  soldiers  with  limited  reading  abilities,  typically  below 
the  6th  grade  level  or  thereabouts. 


The  subjective  judgments  voiced  during  the  interviews  were  in  general  agreement 
with  the  instructors’  later  survey  responses  (see  Table  E.l).  Most  thought  the  soldier 
management  system  was  excellent  or  good  (36  percent  gave  each  rating).  However  five  or 
more  (45%)  gave  ratings  of  "needs  improvement"  or  "poor"  to  the  on-line  lessons,  the 
paper-based  lessons,  and  the  classroom  settings  in  which  they  worked.  Like  the  soldiers, 
about  two-thirds  (7  teachers)  believed  that  JSEP  should  be  used  as  a  part  of  BSEP.  Two 
felt  that  JSEP  should  replace  BSEP. 

Several  respondents  believed  JSEP  was  less  efficient  than  BSEP,  largely  because  JSEP 
did  not  have  the  diagnostic  ability  to  place  soldiers  in  only  those  lessons  that  they 
needed.  One  person  said  that  JSEP  could  not  be  a  stand-alone  program  because  some  of 
the  lessons  did  not  teach  the  content  of  the  prerequisite  competencies  they  covered.  The 
lessons  gave  examples  rather  than  explanations,  in  this  person’s  opinion,  and  additional 
instruction  was  required.  In  one  case  JSEP  was  described  as  slower  than  the  programmed 
McFann-Gray  materials  used  in  BSEP,  and  some  of  the  long  lessons  were  considered  so 
frustrating  that  they  were  described  as  "Chinese  water  torture." 

At  both  of  the  MicroTICCIT  sites  the  respondents  brought  up  the  high  quality 
graphics  and  speed  of  the  system.  They  said  that  students  who  later  used  PLATO  for 
other  instruction  were  often  disappointed  by  its  operation.  One  administrator  also  said 
that  there  were  fewer  repairs,  and  thus  lower  costs,  with  MicroTICCIT  than  with  PLATO. 
On  the  other  hand,  staff  at  both  of  the  MicroTICCIT  sites  complained  that  the  system 
was  not  portable.  PLATO,  with  its  telephone-connected  ports,  could  be  carried  directly  to 
the  units,  while  troops  had  to  be  brought  to  the  MicroTICCIT  classroom. 

Two  peripheral  factors  influenced  the  educators’  assessments  of  JSEP.  One  was  the 
history  of  JSEP’s  development.  The  most  detailed  and  frequent  criticisms  reflected  the 
fact  that  JSEP  was  not  yet  complete  during  f  /88,  which  was  intended  to  be  its 
demonstration  year.  Lessons  and  lesson  tests  contained  errors,  touch  screens 
malfunctioned,  and  there  was  no  diagnostic  test.  This  made  it  difficult  for  educators  (and 
evaluators  who  talked  with  them)  to  decide  whether  they  were  critiquing  the  existant  or 
proposed  program.  Even  though  one  respondent  said  there  had  been  "Quantam  leap 
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improvements"  in  JSEP  recently,  past  difficulties  appeared  to  color  current  opinions  about 
the  program  and  its  future.  At  more  than  one  installation  the  JSEP  staff  cited  frequent 
and  long-term  complaints  about  the  quality  of  JSEP  that  they  felt  had  yet  to  receive  a 
response.  One  site  mentioned  the  difficulty  in  maintaining  continuity  when  it  had  closed 
its  program  for  six  months  because  there  were  no  new  lessons.  One  noted  the 
embarrassment  of  demonstrating  a  lesson  with  errors  to  a  visiting  General.  There  was  a 
remark  that  after  two  years  of  complaining  about  problems  with  the  touch  screens, 
soldiers  were  still  spending  more  time  in  getting  the  equipment  to  work  than  they  were  in 
learning  the  content. 

As  a  result  of  this  history,  many  of  the  education  staff  persons  who  had  the  most 
experience  with  JSEP  seemed  angry  and  doubtful  that  its  present  shortcomings  would  be 
removed.  The  anger  appeared  based  in  some  cases  on  the  feeling  that  local  Education 
Centers  had  invested  good  faith  in  convincing  their  installations  to  pilot  JSEP  and  had 
lost  credibility  when  the  program’s  delivery  did  not  meet  their  expectations.  More 
frequently  the  anger  appeared  to  focus  on  a  perceived  "insensitivity."  Evidences  of  this 
insensitivity  included  not  correcting  lesson  errors,  not  acknowledging  complaints  from  the 
field,  and  not  recognizing  the  value  of  the  soldiers’  time  when  they  participated  in  JSEP. 
On  the  latter  point,  there  were  remarks  that  we  would  characterize  as  almost  bitter  about 
soldiers  being  assigned  lessons  they  did  not  need  in  order  to  test  lessons,  or  being  told  to 
play  computer  games  when  there  were  no  lessons  for  them  to  take.  One  instructor  said 
that  when  he  had  encouraged  counselors  to  refer  soldiers  to  JSEP,  a  counselor  initially 
expressed  unwillingness  to  do  so,  saying,  ’They’re  playing  chess  over  there!"  While  these 
problems  may  reflect  more  than  anything  else  the  different  perspectives  of  educators  and 
program  developers,  they  nonetheless  had  a  strong  effect  on  the  educators’  evaluation  of 
JSEP.  A  self-described  supporter  of  JSEP  reflected  about  as  much  optimism  as  we  heard 
with  the  comment  that:  "It’s  not  neuter  than  gum  -  like  we  thought  —  but  we  can  still 
get  some  use  out  of  it." 

The  second  confounding  factor  was  the  relationship  between  JSEP  and  BSEP.  This 
is  examined  under  the  discussion  of  JSEP’s  perceived  uses,  which  follows. 
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Functions  of  JSEP 


When  educators  talked  about  what  JSEP  was  used  for  it  often  sounded  as  if  they 
were  balancing  two  sets  of  contradictions  in  their  remarks.  The  first  contradiction  was 
between  JSEP’s  facto  and  jure  purposes.  Everyone  interviewed  was  aware  that  JSEP 
was  intended  to  teach  the  prerequisite  competencies  underlying  MOS  job  tasks,  but  as  one 
Educational  Services  Officer  put  it,  "I’ve  never  had  a  soldier  walk  in  saying  he  wanted  to 
increase  his  basic  skills  so  he  could  do  a  better  job."  Two  installations  said  that  they 
marketed  JSEP  specifically  as  a  tool  for  improving  prerequisite  competencies;  one  of  these 
noted  that,  given  the  choice,  soldiers  were  four  times  more  likely  to  enroll  in  BSEP. 

Soldiers  come,  or  are  referred,  to  Education  Centers  to  raise  their  GT  or  SQT  scores. 
Despite  its  intended  use,  the  practicality  is  that  JSEP  is  used  largely  as  GT  improvement. 
Some  educators  find  it  is  used  for  SQT  preparation,  particularly  at  training  installations  in 
which  drill  sergeants  work  outside  their  MOSs  but  still  need  to  pass  the  SQT  to  remain 
eligible  for  reenlistment.  At  one  site  the  JSEP  instructor  had  selected  a  list  of  lessons 
for  common  task  training  and  units  sent  troops  to  JSEP  for  team  training  once  a  week. 

This  was  a  procedure  in  which  soldiers  not  enrolled  in  JSEP  worked  as  a  group  to 
complete  a  selected  lesson. 

Using  JSEP  for  test  preparation  can  lead  to  problems  that  are  exemplified  in  one 
instructor’s  remark  that  JSEP  should  be  used  as  it  was  designed,  but  with  GT  supplements. 
JSEP  was  not  seen  as  capable  of  standing  on  its  own  as  a  GT,  GED,  or  SQT  preparation 
course.  For  these  purposes  it  needs  to  be  augmented  by  instruction  in  reading 
comprehension  improvement  and  by  practice  in  taking  timed  tests.  A  number  of 
respondents  said  that  JSEP  should  be  used  as  a  supplement  to  BSEP  or  as  an  alternative 
for  students  who  did  not  respond  well  to  classroom  instruction. 

A  few  respondents  questioned  whether  JSEP  had  a  clearly  defined  mission.  Several 
thought  that  there  was  little  organization  or  progression  to  the  lessons,  which  seems  a 
reasonable  perception.  The  MOS  justifies  the  choice  of  prerequisite  competencies  and  thus 
structures  a  prescription,  but  the  MOS  is  never  really  seen  in  the  lesson  itself.  More 
common  were  the  respondents  who  pointed  out  that  JSEP  had  the  same  target  audience  as 
BSEP,  that  both  by  definition  taught  job-related  academic  skills,  and  that  JSEP  was  BSEP 
on  a  computer. 
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The  isomorphic  similarities  between  the  two  programs  were  not  the  only  source  of 
comparison.  Because  of  fund  reductions  many  installations  had  cancelled  or  curtailed 
BSEP  classes  and  were  placing  soldiers  with  BSEP  needs  at  JSEP  terminals.  Education 
Center  staff  believed  that  BSEP  was  likely  to  disappear  completely  and  that  JSEP  would 
be  its  only  possible  replacement.  To  the  extent  that  they  saw  a  continuing  need  for 
BSEP,  this  made  it  difficult  to  avoid  comparing  JSEP  with  the  earlier  program.  And,  when 
that  was  done,  JSEP  often  was  seen  as  less  efficient  or  less  complete  than  BSEP. 

Future  of  JSEP 

Instructionally,  the  education  staff  members  believed  JSEP  would  last  in  some  form 
as  long  as  soldiers  needed  BSEP  or  the  Army  used  the  GT  and  SQT  for  career  and 
training  decisions.  This  might  be  interpreted  as  the  belief  that  JSEP  would  last  as  long 
as  it  did  a  job  other  than  that  for  which  it  was  intended.  There  were  recommendations 
that  JSEP  be  considered  for  a  variety  of  uses,  and  that  the  Army  allow  local  Education 
Centers  flexibility  in  what  they  employed  JSEP  for,  and  how  they  used  it.  In  general,  the 
idea  of  self-paced,  individualized  computer-based  instruction  was  seen  as  the  direction  in 
which  Army  education  should  develop.  The  instructional  content  of  JSEP  was  not  seen  as 
capable  of  standing  on  its  own.  It  was  not  possible  to  tell  how  much  these  opinions 
might  change  once  the  program  revisions  are  completed.  And,  at  least  two  people  raised 
the  question  of  whether  it  made  sense  to  separate  education  from  training  at  the  level  of 
BSEP  or  JSEP. 

The  respondents  had  more  questions  than  suggestions  about  JSEP’s  future  when  they 
discussed  logistics.  They  believed  that  the  MicroTICQT  system  would  not  be  used  and 
that  PLATO  might  be  replaced  by  the  Electronic  Information  Dissemination  System  (EIDS), 
in  which  case  the  JSEP  management  system  would  change.  They  did  not  know  who  would 
be  responsible  for  the  costs  of  maintaining  equipment,  updating  lessons  and  writing  new 
ones,  or  hiring  teachers.  On  the  issue  of  updating  lessons,  several  persons  noted  that 
basic  academic  competencies  are  not  likely  to  change  but  that  the  military  context  in 
which  JSEP  presented  them  would.  The  Army,  these  educators  warned,  had  a  low 
tolerance  for  anything  that  smacked  of  obsolescence. 

Requirements  for  instructors.  On  the  question  of  teachers,  interviewees  pointed  out 
that  JSEP  was  not  instructor-free.  This  perception  was  reinforced  by  the  range  of  duties 
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the  instructors  described  in  their  survey  responses.  When  asked  what  percent  of  their 
time  was  used  in  various  tasks,  instructors  reported  averages  for  circulating  and  observing 
soldiers  (28  percent  of  their  time),  record  keeping  (19%),  answering  soldiers’  questions 
about  lessons  (17%),  showing  soldiers  how  to  use  the  program  and  hardware  (13%),  and 
helping  with  program  or  hardware  problems  (10%). 

The  JSEP  instructors  were  also  a  highly  qualified  group.  According  to  their  survey 
responses,  eight  had  been  BSEP  teachers  before  working  with  JSEP  and  all  had  prior 
experience  teaching  military  enlisted  personnel.  All  had  a  baccalaureate  degree  and  three 
had  completed  master’s  programs.  These  qualifications  were  reflected  in  the  requirements 
proposed  for  a  teacher,  which  ranged  from  familarity  with  the  MOS  system  and  inservice 
training  to  a  teaching  certificate  and  experience  in  adult  education.  Estimates  of  how 
many  terminals  a  JSEP  instructor  could  manage  ranged  from  eight  to  15.  Each  respondent 
stressed  that  JSEP  would  not  work  without  a  good  teacher.  One  installation  appeared  to 
have  changed  from  a  philosophy  of  nonintervention  to  having  an  instructor  who  actively 
encouraged  and  assisted  the  students.  With  this  change  the  results,  reputation,  and 
enrollments  in  the  program  improved  dramatically,  in  the  opinion  of  the  Education  Center 
staff. 


Future  instructor  support.  Table  E.l  reports  the  current  JSEP  instructors’  ratings  of 
the  training  they  received  from  Florida  State  University  and  of  the  importance  of 
preparation  in  various  areas.  The  second  volume  of  this  technical  report  (Tart  II 
Curriculum  Review)  discusses  the  quality  of  the  JSEP  Instructor’s  Manual  and  the  Soldier 
Management  System:  Features  and  Functions  that  are  intended  to  serve  as  instructional 
resource  documents.  Without  repeating  information  that  is  available  elsewhere,  we  need  to 
stress  here  that  as  JSEP  is  used  at  a  greater  number  of  sites,  new  and  inexperienced 
instructors  will  be  involved.  In  our  judgment  it  is  crucial  to  the  program’s  success  that 
these  instructors  be  provided  with  inservice  training  on  how  to  use  JSEP,  and  that 
accurate,  easy  to  use  manuals  as  well  as  other  resources  (such  as  telephone  access  to 
experienced  instructors)  be  widely  available  to  them. 

Interviewees’  forecasts.  Projections  on  how  JSEP  would  be  used  two  years  from  now 
were  fairly  standard.  One  educator  thought  that  it  would  be  a  classroom  tool  in  MOS 
training  and  SQT  preparation.  Most,  however,  believed  that  JSEP  would  become  part  of 
the  array  of  lessons  available  on  PLATO.  We  understood  these  projections  to  mean  that 
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JSEP  would  lose  its  identity  as  a  complete  program.  The  individual  lessons  would  remain, 
but  PLATO  laboratory  coordinators  or  basic  skills  instructors  (if  these  remained)  would 
choose  lessons  to  complement  the  needs  of  the  individual  soldier  and  the  specific  task  for 
which  he  or  she  was  being  prepared.  We  also  inferred  that  this  was  the  way  in  which 
the  Education  Center  staff  would  find  JSEP  most  helpful.  This  conclusion  was  based  on 
their  descriptions  of  using  JSEP  as  a  BSEP  supplement  in  the  past  and  on  the  fact  that 
no  one  suggested  how  the  Army  might  prevent  such  a  loss  of  JSEP’s  identity.  This 
seemed  plausible,  considering  the  opinions  voiced  concering  JSEP.  Such  action  would  be 
the  logical  way  to  get  the  most  use  from  a  program  that  is  directed  toward  a  need  which 
is  not  felt  by  its  users,  and  which  is  seen  as  insufficient  in  itself  to  meet  the  needs  they 
consider  important. 

This  chapter  has  described  the  context  in  which  JSEP’s  effects,  and  the  manner  in 
which  it  is  used,  can  be  interpreted.  The  concluding  chapter  summarizes  the  report’s 
findings  and  applies  them  to  recommendations  for  JSEP. 
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Chapter  5.  Summary  of  Findings,  Conclusions,  and 
Recommendations 


This  chapter  draws  upon  the  findings  of  preceding  chapters  to  address  the  evaluation 
questions.  It  does  not  recapitulate  each  individual  finding  from  the  study.  The 
recommendations  with  which  the  chapter  closes  are  based  upon  the  conclusions  drawn  from 
the  findings. 

Findings  and  Conclusions 
Does  JSEP  Teach  What  It  Sets  Out  to  Teach? 

In  general,  yes.  When  soldiers  failed  to  demonstrate  mastery  on  a  lesson  pretest, 
they  usually  passed  the  test  after  instruction.  Examining  the  Common  Core  lessons,  which 
are  prescribed  for  all  soldiers  regardless  of  MOS,  we  found  that  soldiers  passed  83  percent 
of  the  lessons  in  which  they  received  instruction.  However,  some  lessons  were  markedly 
less  "successful"  than  others;  32  lessons  were  failed  by  25  percent  or  more  of  the  soldiers 
who  qualified  for  them  (i.e.,  who  failed  the  pretest).  On  the  other  hand,  39  on-line 
lessons  were  passed  by  100  percent  of  the  soldiers  who  qualified  for  them. 

In  many  cases  JSEP  did  not  have  the  chance  to  teach  content  because  the  soldiers 
already  knew  it.  More  than  half  of  the  soldiers  passed  the  pretest  for  91  (59%)  of  the 
on-line  lessons.  A  total  of  52  lessons  were  passed  on  the  pretest  by  80  percent  or  more 
of  the  soldiers  for  whom  they  were  prescribed. 

What  Are  JSEP’s  Effects  on  Academic  Requirements  for  Jobs? 

There  were  JSEP  prescriptions  for  about  70  percent  of  the  MOSs  reported  by  the 
soldiers  in  this  evaluation  sample.  Soldiers  mastered  most  of  the  prerequisite 


84 


competencies  in  their  prescriptions.  On  the  Common  Core  lessons  they  demonstrated 
mastery  (by  pretesting  out  or  passing  a  test  after  instruction)  in  92  percent  of  the 
quantitative  and  97  percent  of  the  verbal  lessons.  However,  in  most  of  these  lessons  the 
soldiers  demonstrated  mastery  on  the  pretest,  before  taking  the  instruction. 

Soldiers  showed  small  gains  following  participation  in  JSEP  on  a  test  designed  to 
parallel  the  GT:  about  3  points  among  the  group  for  whom  we  had  complete  evaluation 
data.  Instructors  provided  information  on  later  testing  for  about  one-third  of  the 
soldiers.  When  soldiers  took  the  GT  after  JSEP  and  additional  GT  preparation,  scores 
improved  an  average  of  14  points,  roughly  equivalent  to  changes  seen  following  BSEP  in 
earlier  evaluations.  TABE  scores  reported  by  instructors  showed  average  improvement  of 
about  half  a  grade  level  in  reading  and  one  and  one-half  grade  levels  in  mathematics. 
While  there  was  usually  no  additional  instruction  between  the  time  a  soldier  completed 
JSEP  and  the  time  when  the  TABE  posttest  was  administered,  there  is  no  way  of  knowing 
what  learning  opportunities  had  occurred  between  the  time  of  the  TABE  pretest  and  the 
soldier’s  enrollment  in  JSEP. 

Supervisors  generally  had  not  observed  any  change  in  soldier’s  work  performance 
following  participation  in  JSEP.  The  relatively  few  who  did  notice  a  difference  found 
soldiers  to  be  more  self-confident  and  less  impulsive  in  making  decisions.  However,  many 
of  the  JSEP  graduates  and  their  supervisors  who  were  interviewed  cited  specific  test  score 
improvements  they  attributed  to  the  program. 

Does  JSEP  Work  as  Intended? 

The  answer  to  this  is  a  qualified  "yes:”  in  some  ways,  for  this  possibly  atypical 
evaluation  sample.  When  we  look  at  the  Common  Core  lessons,  soldiers  pretested  out  of 
64  percent  of  them  and  failed  6  percent.  This  can  be  interpreted  as  JSEP’s  "working"  for 
the  remaining  30  percent.  However,  the  soldiers  in  this  sample  had  reading  and 
mathematics  achievement  levels  above  what  might  typically  be  expected  for  B;  3P  classes: 
10.1  and  8.7  grade  levels  in  each  subject  respectively.  This  explains  in  large  part  why  so 
many  soldiers  pretested  out  of  so  many  JSEP  lessons.  Although  the  sample  did  not 
include  those  with  fifth  or  sixth  grade  reading  levels  whom  the  instructors  feared  would 
not  be  able  to  use  JSEP,  the  evidence  was  that  the  lower  a  soldier’s  reading,  math,  and 
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GT  test  scores,  the  less  likely  the  soldier  was  to  pass  JSEP  lessons  on  pretest  and  the 
more  likely  the  soldier  was  to  fail  lessons  after  instruction. 

About  half  of  the  soldiers  were  able  to  complete  the  on-line  lessons  in  the  Common 
Core  prescription  in  43  hours  of  instruction  or  less,  but  there  were  wide  differences  in 
the  time  needed  by  the  fastest  and  slowest  soldiers  in  every  lesson.  And  we  have  no  time 
figures  for  the  paper  lessons,  since  this  information  was  not  collected  by  the  management 
system.  The  JSEP  management  system  on  PLATO  was  adapted  for  the  evaluation  in  such 
a  way  that  soldiers  were  given  a  maximum  of  two  exposures  to  the  instructional  content: 
once  through  the  short  lesson,  and  once  through  the  long  lesson  if  that  were  needed  (and 
available).  The  MicroTICCIT  system  was  not  limited  by  this  adaptation,  and  soldiers 
attempted  the  content  of  some  lessons  four  or  five  times.  Thus  the  time-to-complete 
figures  estimated  from  this  evaluation  are  undoubtedly  conservative. 

Unfortunately,  we  were  unable  to  observe  cases  in  which  JSEP  was  undeniably 
inappropriate.  Instructors  felt  that  the  program  would  not  work  with  soldiers  having  poor 
reading  skills,  and  very  few  of  these  were  included  in  the  evaluation.  Among  the  few 
dropouts  we  interviewed,  inappropriateness  appeared  more  a  matter  of  temperament  than 
reading  skills.  A  small  number  of  soldiers  found  concentrating  alone  on  a  computer 
terminal  to  be  boring,  lonely,  or  frustrating. 

Soldiers  and  Education  Center  staff  typically  rated  JSEP  high  on  interest  and 
enjoyability.  Both  groups  felt  that  individualized,  computer-based  instruction  was  a  good 
approach  to  use  with  soldiers  who  had  varying  abilities  and  needs,  and  who  appreciated 
the  chance  to  make  mistakes  in  private.  More  than  half  of  the  instructors  rated  the 
content  of  the  lessons  as  poor  or  needing  improvement,  and  educators  generally  felt  the 
program  did  a  better  job  teaching  quantitative  than  verbal  skills.  Some  felt  that  JSEP 
was  more  effective  than  BSEP  in  teaching  basic  skills.  However,  more  instructors  and 
coordinators  felt  that  JSEP  was  less  effective  than  BSEP,  but  could  still  be  a  useful 
complement  to  the  traditional  classroom  program. 

In  terms  of  efficiency,  almost  all  of  the  Education  Center  staff  referred  JSEP 
graduates  to  additional  test  preparation.  They  did  not  see  JSEP  as  sufficient  for  GT  or 
SQT  preparation,  which  were  almost  exclusively  perceived  as  the  needs  to  be  met. 

Students  and  staff  also  insisted  that  JSEP  required  an  instructor,  and  from  their  estimates 
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and  our  observations  it  appears  that  a  JSEP  instructor  can  handle  no  more  students  at  a 
given  time  than  can  a  traditional  BSEP  classroom  teacher.  We  conclude  that  if  JSEP  is 
envisioned  as  a  replacement  for  BSEP  it  will  serve  a  more  limited  audience,  be  equally 
costly  in  terms  of  instructional  staff  and  student  time,  and  need  supplementation  in 
reading,  vocabulary,  composition,  and  test-taking  instruction.  Under  these  conditions  JSEP 
may  be  almost  as  effective  in  improving  GT  performance  as  is  BSEP. 


Unresolved  Issues,  Untested  Questions 

The  preceding  conclusion  is  not  a  definitive  judgment  of  whether  JSEP  "works," 
because  although  GT  improvement  was  the  goal  of  virtually  all  participants  it  was  not  the 
goal  of  JSEP.  There  are  basic  questions  to  be  answered  about  what  JSEP  is  supposed  to 
do,  and  for  whom  it  is  intended  to  do  it,  before  the  program’s  worth  can  be  decided. 

Determining  the  Audience  for  JSEP 

The  target  population  for  JSEP  has  not  been  defined  clearly.  The  program  goals  and 
eligibility  criteria  that  delineated  the  target  population  were  first  officially  stated  in 
correspondence  from  the  Soldier  Education  Division  in  August  198 7.  JSEP  had  been  under 
development  for  she  years  before  these  criteria  were  made  explicit,  and  by  January  1988 
there  was  still  some  uncertainty  as  to  how  they  fit  with  the  program.  The  stated  goals, 
for  example,  were  to  provide  soldiers  with  the  academic  competencies  required  for  job 
performance,  skill  qualification,  and  career  growth.  Eligibility  was  defined  through  Army 
classification  test  scores  (particularly  as  these  related  to  reenlistment  requirements); 
undefined  commander  recognition  of  performance  deficiencies  that  might  be  attributable  to 
academic  weakness;  and  performance  on  JSEP  pretests  that  did  not  yet  exist  in  usable 
form. 

Referral  on  the  basis  of  job  performance,  and  performance  on  a  JSEP  placement  test 
were  compatible  with  the  concepts  underlying  JSEP’s  design.  But  this  leads  to  additional 
questions  about  when  the  program  should  be  introduced.  If  JSEP  teaches  the  competencies 
prerequisite  to  learning  an  MOS,  then  it  would  be  most  useful  before  MOS  training  is 
completed.  But  the  selection  criteria  imply  that  referral  will  not  take  place  until  a 
soldier  is  on  the  job,  performing  unacceptably. 
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Furthermore,  using  classification  test  score  criteria  as  determinants  of  eligibility 
implies  a  different  goal  for  the  program:  increasing  classification  test  scores,  with 
primary  emphasis  on  the  General  Technical  (GT)  composite.  This  would  be  readily 
accepted  by  JSEP’s  users.  The  perceived  importance  of  the  GT  by  soldiers,  their 
supervisors,  and  educational  staff  cannot  be  overemphasized.  Improving  the  GT  score  is 
the  first  and  often  only  goal  of  soldiers  enrolling  in  JSEP  and  the  commanders  who  refer 
them  to  the  program.  Unless  there  is  a  sizable  and  demonstrable  positive  relationship 
between  increases  in  GT  score  and  improvement  in  the  academic  competencies  JSEP 
defines  as  prerequisites  for  job  performance,  skill  qualification,  and  career  growth,  the 
stated  and  implied  goals  for  JSEP  may  not  be  compatible.  JSEP  could  conceivably  do  an 
excellent  job  of  improving  soldiers’  academic  prerequisite  competencies  but  have  little 
discernible  effect  upon  their  classification  test  scores. 

However,  if  GT  scores  did  improve,  this  would  lead  to  the  alternate  problem  of 
explaining  how  this  related  to  JSEP  instruction.  To  our  knowledge  there  is  no  solid 
empirical  evidence  that  definitively  links  proficiency  in  academic  skills,  such  as  those 
taught  by  JSEP,  to  Army  classification  test  scores.  Fragmentary  data  were  reported, 
showing  increases  in  GT  scores  following  JSEP  combined  with  other  remedial  instruction 
directed  toward  GT  improvement  Unfortunately,  these  data  cannot  tell  how  much  of  the 
test  score  improvement  was  attributable  to  JSEP  and  how  much  was  explained  by  other 
instruction.  We  could  not  assume  that  JSEP  caused  all  or  most  of  the  change,  particularly 
in  view  of  AIR’s  earlier  evaluation  findings  that  a  number  of  BSEP  programs  were 
associated  with  substantial  increases  in  GT  scores  (Hahn,  et  al.,  1986). 

Determining  a  Function  for  JSEP 

One  Education  Center  staffer  has  already  been  quoted  to  the  effect  that  no  soldier 
ever  walked  into  his  office  and  asked  for  education  in  order  to  do  a  better  job.  If  the 
function  of  JSEP  is  to  teach  prerequisite  competencies  that  will  improve  MOS 
performance,  another  Education  Center  interviewee  may  reflect  a  common  belief  in  saying 
"JSEP  provides  us  with  an  answer  to  a  problem  we  haven’t  got.”  That  remark  could  have 
been  intended  as  a  joke,  but  it  makes  the  serious  point  that  if  JSEP  does  not  meet  a 
perceived  need  it  will  not  be  used. 
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Instructors  and  educational  administrators  at  the  pilot  sites  said  that  they  were 
aware  JSEP  was  not  a  GT  improvement  program.  They  also  said  they  realized  it  was  not 
a  replacement  for  BSEP.  This  explanation  held  that  BSEP  was  meant  to  disappear  as 
higher  recruiting  standards  enlisted  fewer  BSEP-eligible  soldiers;  JSEP  might  look  like 
BSEP  and  might  be  introduced  at  the  same  time  that  BSEP  was  phased  out,  but  the  two 
programs  had  different  goals  and  target  audiences.  Finally,  the  educators  realized  that 
JSEP  was  not  job  training.  They  were  able  to  make  the  distinction  between  prerequisite 
and  job  competencies,  although  many  volunteered  that  the  distinction  was  not  easy  to 
explain  to  others. 

In  short,  there  was  good  agreement  about  what  JSEP  was  not,  but  this  included  the 
functions  for  which  existing  programs  were  valued:  GT  or  SQT  improvement,  general 
academic  growth  as  the  first  step  in  further  formal  education,  or  job  performance 
improvement. 

In  the  judgment  of  those  conducting  this  evaluation,  if  JSEP  does  not  have  a  clearly 
defined  and  widely  accepted  purpose  it  will  not  be  used.  We  believe  that  if  these  issues 
are  not  resolved  JSEP  will  cease  to  exist  as  a  distinct  program,  and  will  become  part  of 
the  vast  collection  of  PLATO  lessons  from  which  an  instructor  or  learner  can  choose  a 
customized  selection.  This  would  not  seem  an  effective  use  of  the  resources  that  have 
been  invested  in  the  program. 

Recommendations 

These  conclusions  about  JSEP’s  unresolved  ambiguities,  as  well  as  other  findings  in 
the  JSEP  evaluation,  have  led  us  to  several  recommendations  concerning  the  program’s 
future. 


1.  The  Soldier  Education  Division  should  initiate  discussion  with  Education  Center 
staff  members  concerning  current  and  future  needs  in  basic  skills  instruction, 
and  how  JSEP  can  be  used  or  adapted  to  meet  these  needs.  Such  discussion 
should  lead  to  specification  of,  and  consensus  about,  the  purposes  and 
appropriate  audiences  for  JSEP.  We  recommend  discussion  rather  than  direction 
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because  we  believe  that  unless  JSEP  addresses  needs  felt  to  be  important  by  its 
potential  users  it  will  not  be  used. 

2.  All  planned  lesson  and  lesson  test  revisions  should  be  implemented  at  this  time. 
Lessons  for  which  anomalous  results  (e.g.,  unusually  high  or  low  pretest  passing 
or  lesson  failure  rates)  are  reported  in  this  study  should  be  examined  carefully 
by  curriculum  designers  to  determine  if  further  revision  is  warranted.  Revision 
should  be  followed  by  at  least  one  more  review  to  ensure  that  all  errors  have 
been  corrected.  Support  (inservice  training  and  resource  manuals)  for  future 
instructors  should  be  in  final  form  before  the  program  is  considered  complete. 

3.  The  Soldier  Education  Division  should  consider  the  question  of  JSEP  maintenance. 
Education  staff  persons  agreed  widely  that  even  if  the  basic  skills  remained 
stable  over  time  the  MOS  requirements  or  context  in  which  these  skills  were 
taught  would  change,  and  that  the  appearance  of  obsolescence  would  prevent 
instructors  from  using  the  program. 

4.  The  lesson  pretest  option  should  be  retained  until  the  Army  Research  Institute  is 
perfectly  satisfied  that  the  diagnostic  test  currently  under  development  is  equally 
accurate  at  identifying  competencies  potential  students  have  already  mastered. 

5.  The  JSEP  soldier  management  system  should  be  revised  to  notify  soldiers  about 
which  lesson  test  items  have  been  answered  incorrectly.  Additional  test  items 
should  be  developed  for  all  lessons  so  that  soldiers  retaking  a  lesson  test  will 
not  necessarily  repeat  the  same  items.  Further,  the  cost  of  revising  the  tests  so 
that  greater  flexibility  in  answering  (or  less  ambiguity  in  directions  about  how 
to  answer)  is  allowed  should  be  studied. 

6.  The  Soldier  Education  Division  should  consider  allowing  and  perhaps  encouraging 
extensive  instructor  intervention  in  JSEP.  Correcting  for  program  errors, 
encouraging  students  to  persevere,  providing  alternative  instruction  when  the 
computer-based  program  does  not  "work"  for  a  student,  and  supplementing  with 
instruction  directed  toward  a  soldier’s  individual  needs,  appear  to  be  important 
factors  in  success. 
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7.  The  Soldier  Education  Division  (preferably  with  input  from  local  Education 
Centers)  should  consider  how  it  wants  JSEP  to  be  used  and  should  promulgate 
guidelines  to  this  effect.  Policy  should  consider  whether  JSEP  is  a  unified 
program  or  a  collection  of  lessons,  how  and  if  prescriptions  are  to  be 
determined,  whether  paper  lessons  are  to  be  required,  if  instructors  should 
exempt  soldiers  from  lessons  they  judge  to  be  overly  difficult  or  irrelevant,  and 
what  criteria  are  to  be  used  in  determining  which  soldiers  should  enroll  in  JSEP. 

8.  Finally,  we  must  point  out  that  this  cannot  be  considered  a  summative  evaluation 
of  JSEP:  the  program  was  not  examined  in  its  final  form,  the  conditions  under 
which  it  operated  were  not  typical,  and  the  objectives  against  which  it  should  be 
measured  had  not  been  determined.  We  urge  the  Soldier  Education  Division  to 
institute  some  procedure  for  monitoring  the  effects  and  efficiency  of  JSEP; 
preferably  on  a  regular  basis,  but  if  this  is  not  possible,  at  least  once  when  the 
finished  program  is  in  normal  operation. 
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APPENDIX  A 


JSEP  Lessons  and  Tests:  Type,  Status,  Standards 


Table  A.l.  JSEP  Lessons  and  Tests:  Type,  Status,  Standards 
(During  Evaluation,  19  Jan--15  Apr  1988) 


Lesson  Name 

Lesson 

Type 

f L.D.P.M1 

Revision 
Complete 
DRL  SDL 

Number 

Test 

Items 

Pass 

Score 

(%) 

Numberinq  &  Countinq 
la.  Match  numeral  with  name 

L 

Y 

N 

20 

80 

lb. 

Write  in  sequence 

L 

N 

N 

12 

83 

lc. 

Identify  number  before,  between 

L 

Y 

N 

7 

86 

Id. 

Greater/lesser  than 

L 

N 

N 

10 

80 

le. 

Ordinal  position 

L 

N 

Y 

11 

81 

If. 

Place  value 

L 

N 

N 

13 

76 

lg. 

Round  whole  or  decimal  number 

L 

N 

Y 

10 

80 

lh. 

Count  by  1,2,5,10,  etc. 

D 

N 

- 

12 

83 

li. 

Match  number  with  scale  intervals 

D 

Y 

- 

20 

80 

Linear.  Weiqht  &  Volume  Measures 

2a.  Interpret  linear  scale  markings 

L 

Y 

Y 

8 

75 

2b. 

Ident  US  standard  &  metric  measures 

L 

Y 

Y 

15 

80 

2c. 

Measure  lengths 

L 

Y 

Y 

10 

80 

2d. 

Ident  measures:  weight,  pressure,  torque 

D 

Y 

- 

9 

77 

2e. 

Ident  measures:  volume 

D 

Y 

- 

16 

81 

2f . 

Measure  with  non-numeric  calibrated 

D 

Y 

- 

9 

77 

2g. 

Estimate  size,  distance 

L 

Y 

Y 

9 

77 

Decree  Measures 

3a.  Ident  degrees  &  mils,  angles  or  temp 

L 

Y 

Y 

14 

78 

3b. 

Estimate  angle  <  180  degrees 

L 

Y 

Y 

10 

80 

3c. 

Interpret  azimuths,  range  0-6400  mils 

L 

Y 

Y 

8 

87 

3d. 

Interpret  azimuths,  range  0-360  degree 

L 

Y 

Y 

8 

87 

Time- 

4a. 

-Tel  1 inq  Measures 

Tell  time:  digital,  analog,  24  hour 

L 

Y 

Y 

12 

83 

4b. 

Use  clockface  positions  for  direction 

L 

Y 

Y 

10 

80 

4c. 

Estimate  seconds,  minutes,  parts  hour 

L 

Y 

Y 

10 

80 

4d. 

Equivalent  dates,  Gregorian  &  Julian 

L 

Y 

Y 

10 

80 

4e. 

Convert  time  to  hours,  lOths  hours 

D 

Y 

- 

10 

80 

4f . 

Convert  to  Zulu  (Greenwich  Mean)  time 

L 

Y 

Y 

10 

80 

Diagnostic  Review  Lesson  (DRL)  only 
L-Both  Skill  Development  (SDL)  and 
Diagnostic  Review  (DRL)  Lessons 
Mixed  format 
N«No 


P=Paper  Lesson 

X=Mixed  format,  paper  segment 
not  written 
Y=Yes 


A-l 


Table  A.l.  (Cont.)  JSEP  Lessons  and  Tests:  Type,  Status,  Standards 
(During  Evaluation,  19  Jan--15  Apr  1988) 


Lessor 

Name 

Lesson 

Type 

(L,D,P,M) 

Revision 
Complete 
DRL  SDL 

Number 

Test 

Items 

Pass 

Score 

(%) 

Gaae  Measures 

5a.  Read  4  interpret  gage 

L 

Y 

Y 

14 

78 

5b. 

Read  4  interpret  display  read-out 

L 

Y 

N 

10 

80 

5c. 

Read  4  interpret  gage  with  color 

L 

Y 

N 

10 

80 

5d. 

Read  4  interpret  scales  with  (+/-) 

L 

Y 

N 

12 

83 

5e. 

Read  4  interpret  multiscale  gage 

L 

N 

N 

10 

80 

5f . 

Match  gage  reading  to  spec 

L 

Y 

N 

10 

80 

5g. 

Read  4  interpret  unnumbered/unmarked 

L 

N 

N 

10 

80 

5h. 

Read  4  interpret  fluctuating  gage 

L 

Y 

N 

8 

87 

5i . 

Match  specs  by  manipulation,  alinement 

L 

N 

N 

10 

80 

Soatial 

6a.  Identify  directions  tools  may  move 

L 

Y 

N 

15 

73 

6b. 

Manipulate  objects  to  aline,  etc. 

L 

N 

N 

13 

76 

6c. 

Interpret  spatial  from  2-dimension 

L 

N 

N 

15 

80 

6d. 

Relate  symbols  to  systems,  components 

D 

N 

- 

9 

77 

Lines 

7a. 

Ident  points,  lines,  segments,  rays 

L 

Y 

Y 

10 

80 

7b. 

Ident  vertical,  horizontal,  diagonal 

L 

Y 

Y 

10 

80 

7c. 

Ident  intersecting,  divergent,  etc. 

L 

Y 

Y 

8 

87 

7d. 

Superimpose  lines 

D 

Y 

- 

5 

80 

7e. 

Draw  1 ines 

P 

Y 

- 

5 

80 

PI anes 
8a. 

> 

Match  plane  geometric  4  common  shapes 

L 

Y 

Y 

13 

76 

8b. 

Ident  characteristics  geometric  shapes 

D 

Y 

- 

8 

75 

8c. 

Apply  shape  terms  to  objects 

L 

Y 

Y 

10 

80 

8d. 

Match  patterns,  actual  size  4  drawings 

L 

Y 

Y 

9 

88 

8e. 

Ident  figure  orientation 

D 

Y 

• 

8 

75 

D*Diagnostic  Review  Lesson  (DRL)  only 
L-Both  Skill  Development  (SDL)  and 
Diagnostic  Review  (DRL)  Lessons 
M=Mixed  format 
N=No 


P=Paper  Lesson 

X=Mixed  format,  paper  segment 
not  written 
Y=Yes 


Table  A.l.  (Cont.)  JSEP  Lessons  and  Tests:  Type,  Status,  Standards 
(During  Evaluation,  19  Jan--i5  Apr  1988) 


Lesson 

Revision 

Number 

Pass 

Type 

Complete 

Test 

Score 

Lesson 

i  Name 

IL.D.P.M) 

PRL 

SDL 

Items 

WkAWk 

Anqles 

;  4  Trianqles 

9a. 

Ident  angles 

L 

Y 

Y 

18 

77 

9b. 

Ident  types  of  angles  (vertical,  etc.) 

D 

Y 

- 

10 

80 

9c. 

Ident  types  of  triangles 

L 

Y 

Y 

10 

80 

9d. 

Ident  altitudes  &  bisectors 

L 

Y 

Y 

14 

78 

9e. 

Name  angles,  using  letters  4  numbers 

D 

Y 

- 

13 

76 

Sol  ids 

10a. 

Recognize  4  match  names  with  figures 

L 

N 

N 

9 

77 

Terminoloqy 

11a. 

Ident  shape  4  position  terms 

P 

N 

- 

20 

75 

lib. 

Ident  spatial  orientation  terms 

P 

N 

- 

30 

80 

Addition  4  Subtraction 

12a. 

Whole  numbers,  no  carrying 

L 

Y 

Y 

16 

81 

12b. 

Whole  numbers,  carrying 

L 

Y 

Y 

12 

66 

12c. 

Decimals,  carrying 

Positive  4  negative  numbers 

L 

Y 

Y 

10 

80 

1 2d . 

L 

Y 

Y 

10 

80 

12e. 

24-hour  time 

L 

Y 

Y 

10 

80 

12f . 

Increments  on  measuring  instruments 

L 

Y 

Y 

16 

81 

12g. 

Linear,  dry,  liquid,  degree  measures 

L 

Y 

Y 

11 

81 

12h . 

Estimate  sum  or  difference 

L 

Y 

Y 

15 

80 

MultiDl ication  4  Division 

13a. 

Whole  numbers 

L 

Y 

Y 

21 

76 

13b. 

Whole  4  decimal  numbers 

L 

Y 

Y 

10 

80 

13c. 

Decimals  in  divisor  4  dividend 

L 

Y 

Y 

10 

80 

13d. 

Negative  4  positive  numbers 

L 

Y 

Y 

14 

85 

13e. 

Estimate  product  or  quotient 

L 

Y 

Y 

10 

80 

D=Diagnostic  Review  Lesson  (DRL )  only 
L=*Both  Skill  Development  (SDL)  and 
Diagnostic  Review  (DRL)  Lessons 
M=Mixed  format 
N*No 


P=Paper  Lesson 

X=Mixed  format,  paper  segment 
not  written 
Y=Yes 
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Table  A.l.  (Cont.)  JSEP  Lessons  and  Tests:  Type,  Status,  Standards 
(During  Evaluation,  19  Jan--15  Apr  1988) 


Lesson  Name 

Lesson 

Type 

(L,D, P,M) 

Revision 
Complete 
DRL  SDL 

Number 

Test 

Items 

Pass 

Score 

(%) 

Fractions/Decimal s 

14a.  Estimate  fractional  length,  area,  vol 

L 

Y 

Y 

10 

80 

14b. 

Reduce  to  lowest  terms 

L 

Y 

Y 

10 

80 

14c. 

Convert  to  decimals  &  vice-versa 

L 

Y 

Y 

15 

80 

14d. 

Convert  decimals/percents  to  fractions 

L 

Y 

Y 

12 

67 

14e. 

Add  4  subtract  fractions 

L 

Y 

Y 

11 

72 

1 4  f . 

Multiply  &  divide  fractions 

L 

Y 

Y 

20 

80 

14g. 

Estimate  fraction  sum,  product,  quotient 

L 

Y 

Y 

10 

80 

Geometry 

15a.  Draw  plane  geometric  figures 

D 

N 

10 

80 

15b. 

Match  geometric  figures  with  names 

L 

Y 

N 

18 

83 

15c. 

Label  objects  4  figures 

D 

Y 

- 

14 

71 

15d . 

Use  protractor 

L 

Y 

Y 

10 

80 

15e. 

Draw  perpendicular  lines,  protractor 

M 

Y 

X 

not  available 

1 5  f . 

Compute  area,  perimeter  rectangle 

L 

Y 

N 

10 

80 

15g. 

Radius,  area,  circumference  circle 

L 

N 

N 

10 

80 

15h . 

Measure  rectangular  solids 

L 

Y 

N 

10 

80 

1 5  i . 

Solve  geometric  problems  with  formulas 

L 

N 

Y 

25 

80 

1 5  j . 

Solve  oscilloscope  readouts 

L 

N 

N 

10 

80 

Combination  of  Processes 

16a.  Locate  center  of  object 

D 

Y 

Y 

5 

80 

16b. 

Compute  averages 

L 

Y 

Y 

12 

83 

16c. 

Use  all  processes,  whole  4  mixed 

L 

Y 

Y 

10 

80 

16d. 

Use  all  processes,  units  measurement 

L 

Y 

Y 

20 

80 

16e. 

Use  info  from  charts,  graphs,  etc. 

L 

Y 

Y 

20 

80 

1 6  f . 

Solve  conversion  problems 

L 

Y 

Y 

10 

80 

16g . 

Ratio  4  proportion  problems 

L 

Y 

Y 

7 

85 

16h . 

Use  all  processes,  word  problems 

L 

Y 

Y 

10 

80 

D=Diagnostic  Review  Lesson  (DRL)  only 
L=*Both  Skill  Development  (SDL)  and 
Diagnostic  Review  (DRL)  Lessons 
M=Mixed  format 
N*No 


P=Paper  Lesson 

X=Mixed  format,  paper  segment 
not  written 
Y=Yes 
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Table  A.l.  (Cont.)  JSEP  Lessons  and  Tests:  Type,  Status,  Standards 
(During  Evaluation,  19  Jan--15  Apr  1988) 


Lesson 

Revision 

Number 

Pass 

Type 

Complete 

Test 

Score 

Lesson  Name 

(L,D, P,M) 

_DRL_ 

SDL 

Items 

KM 

Graohina  in  the  Coordinate  Plane 

17a. 

Ident  grid  coordinates  military  map 

L 

Y 

Y 

10 

80 

17b. 

6-digit  coordinates,  map  line  intersect 

P 

Y 

- 

10 

80 

17c. 

Plot  point,  distance  &  direction  given 

L 

Y 

Y 

6 

83 

17d. 

8-digit  coordinates,  map  line  intersect 

P 

Y 

- 

10 

80 

Algebra 

N 

18a. 

Simple  equations,  one  unknown 

L 

N 

10 

80 

18b. 

Derive  equivalent  equations 

L 

N 

N 

10 

80 

18c. 

Use  calculator,  find  power,  square  root 

L 

N 

N 

10 

80 

Triqonometry 

19a. 

Use  tables  trig  functions  (mils) 

D 

Y 

- 

10 

80 

19b. 

Log  tables  to  multiply,  divide 

P 

Y 

- 

9 

77 

19c. 

Trig  to  solve  geometry,  triangle  side 

L 

Y 

Y 

8 

87 

19d. 

Trig  function  tables  (degrees) 

D 

Y 

- 

8 

87 

Procedural  Directions 

25a. 

Follow  directions  (read,  observe,  etc.) 

P 

N 

- 

10 

80 

25b. 

Select  text,  materials  to  complete  task 

P 

N 

- 

17 

76 

25c. 

Follow  details,  sequence  task  activity 

P 

N 

- 

9 

77 

25d . 

Determine  message  of  job  material 

P 

N 

- 

13 

76 

25e. 

Select  appropriate  decision 

P 

N 

- 

8 

75 

25f . 

Synthesize  info  to  complete  task 

P 

N 

- 

7 

71 

Vocabul ary 

26a. 

Recognize  meaning  common  words 

L 

Y 

Y 

30 

80 

26b. 

Recognize  aircraft,  tank  words 

L 

Y 

Y 

22 

77 

26c. 

Ident  meaning  from  context 

L 

Y 

Y 

10 

80 

26d. 

Meaning  contractions,  abbrevs,  acronym 

L 

Y 

Y 

15 

80 

26e . 

Meaning  figurative,  idiomatic  terms 

L 

Y 

Y 

13 

84 

26  f . 

Communication,  navigation  words 

D 

N 

- 

21 

80 

26g . 

Rifle,  survival  words 

D 

N 

* 

17 

82 

D-Diagnostic  Review  Lesson  (DRL)  only 
L*8oth  Skill  Development  (SDL)  and 
Diagnostic  Review  (DRL)  Lessons 
M=Mixed  format 
N=No 


P=Paper  Lesson 

X=Mixed  format,  paper  segment 
not  written 
Y=Yes 


A-5 


Table  A.l.  (Cont.)  JSEP  Lessons  and  Tests:  Type,  Status,  Standards 
(During  Evaluation,  19  Jan--15  Apr  1988) 


Lesson  Name 

Lesson 

Type 

fL.D.P.Ml 

Revi sion 
Complete 
DRL  SDL 

Number 

Test 

Items 

Pass 

Score 

(%) 

Reference  Skills 

27a.  Locate  documents  by  code  4  title 

L 

Y 

Y 

10 

80 

27b. 

Locate  4  file,  alpha  4  numeric 

L 

N 

N 

12 

83 

27c. 

Use  table  contents,  index,  glossary 

L 

N 

Y 

11 

81 

27d. 

Locate  title,  page,  etc.,  needed 

L 

N 

N 

16 

81 

27e. 

Skim  or  scan  for  info 

L 

N 

Y 

10 

80 

2  7  f . 

Use  cross  references 

L 

Y 

N 

10 

80 

27g. 

Organize  info,  multiple  sources 

L 

N 

N 

not  available 

Tabl es/Charts 

28a.  Obtain  fact  from  two-column  table 

L 

Y 

N 

9 

77 

28b. 

Obtain  fact  from  table  intersection 

M 

N 

- 

8 

75 

28c. 

Use  complex  table,  cross  references 

L 

N 

N 

8 

75 

28d. 

Use  tables  to  locate  malfunction 

L 

N 

N 

11 

81 

Ill ustrations 

29a.  Ident  details  from  illustration 

L 

N 

Y 

16 

81 

29b. 

Ident  details  from  key,  legend,  list 

L 

N 

Y 

9 

77 

29c. 

Use  cross-sectional  view  for  decision 

L 

N 

Y 

12 

83 

29d . 

Use  three-dimensional  projection 

D 

N 

- 

6 

83 

29e. 

Use  illustration  to  follow  directions 

L 

N 

Y 

10 

80 

29f . 

Integrate  visual  info,  many  sources 

M 

Y 

X 

not  available 

29g. 

Use  map  ident  terrain,  location 

lesson 

not 

available 

FI  ow 
30a. 

Charts 

Ident  meanings  flow  chart  symbols 

D 

Y 

6 

83 

30b. 

Use  flow  chart,  procedural  decision 

D 

Y 

- 

8 

87 

30c. 

Use  to  ident  organization  members 

D 

Y 

11 

81 

Diagnostic  Review  Lessc..  (DRL)  only 
L=Both  Skill  Development  (SDL)  and 
Diagnostic  Review  (DRL)  Lessons 
Mixed  format 
N=No 


P=Paper  Lesson 

X=Mixed  format,  paper  segment 
not  written 
Y=Yes 


Table  A.l.  (Cont.)  JSEP  Lessons  and  Tests:  Type,  Status,  Standards 
(During  Evaluation,  19  Jan--15  Apr  1988) 


Lesson  Name 

Lesson 

Type 

(L.D.P.M) 

Revision 
Complete 
DRL  SDL 

Number 

Test 

Items 

Pass 

Score 

(%} 

Schematics 

31a.  Ident/locate  subsystems  block,  etc. 

L 

Y 

Y 

10 

80 

31b. 

Ident  components,  signal  paths 

L 

Y 

Y 

20 

80 

31c. 

Trace  circuit  connections  in  schematic 

D 

Y 

- 

5 

80 

3  Id . 

Ident  faulty  components/troubleshoot 

L 

Y 

Y 

18 

77 

31e. 

Ident  symbols:  components,  signal  paths 

L 

Y 

Y 

9 

77 

Forms 

32a. 

Locate  block  to  enter  info 

L 

N 

N 

12 

83 

32b. 

Transfer  data  onto  proper  section 

L 

Y 

N 

12 

75 

32c. 

Enter  selected  info  onto  form 

L 

N 

N 

19 

84 

32d. 

Write  descriptive  account 

M 

N 

X 

not  available 

32e. 

Use  completed  form  to  find  info 

L 

N 

Y 

12 

66 

Note- 

33a. 

Takinq 

Record  essential  info 

P 

N 

45 

80 

33b. 

Accuracy  &  precision  recording  info 

P 

N 

- 

10 

80 

33c. 

Record  info  as  sentence 

P 

Y 

- 

10 

80 

33d. 

Record  info,  more  than  one  sentence 

P 

N 

- 

16 

75 

Outlininq  (Tooic  or  Sentence! 

34a.  Ident  main  idea 

D 

N 

14 

78 

34b. 

Recognize  titles  for  outline  sections 

D 

N 

- 

9 

77 

34c. 

Select  supporting  details 

D 

N 

- 

31 

77 

34d . 

Numbers  &  letters  to  label  topics 

D 

N 

- 

5 

80 

34e. 

Write  training  outline 

P 

N 

- 

not  available 

Diagnostic  Review  Lesson  (DRL)  only 
L»Both  Skill  Development  (SDL)  and 
Diagnostic  Review  (DRL)  Lessons 
Mixed  format 
N=No 


P=Paper  Lesson 

X=Mixed  format,  paper  segment 
not  written 
Y=Yes 
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Table  A.l.  (Cont.)  JSEP  Lessons  and  Tests:  Type,  Status,  Standards 
(During  Evaluation,  19  Jan--15  Apr  1988) 


Lesson  Name 

Lesson 

Type 

a.D.P.M) 

Revision 
Complete 
DRL  SDL 

Number 

Test 

Items 

Pass 

Score 

(%) 

ReDort  Writinq 

35a.  Identify  objectives,  audiences,  etc. 

P 

N 

8 

88 

35b. 

Answer  who,  what,  when,  where,  how 

P 

N 

- 

10 

80 

35c. 

Select  relevant  details 

P 

N 

- 

20 

70 

35d. 

Generate  report,  sequence  events 

P 

N 

- 

40 

80 

35e. 

State  general  impressions  of  event 

P 

N 

- 

20 

80 

3  5  f . 

Write  report 

P 

N 

- 

8 

75 

35g. 

Summarize  events  &  dialog 

P 

N 

- 

6 

83 

35h . 

Summarize  major  points 

P 

N 

- 

29 

72 

35 i . 

Write  report  to  justify  action 

P 

N 

- 

16 

75 

35  j. 

Generate  report  by  format 

P 

N 

10 

80 

Edit 

36a. 

ioa 

Spell  common  words 

L 

N 

N 

30 

80 

36b. 

Spell  task-related  words 

L 

N 

N 

30 

80 

36c. 

Ident  words  needing  capitalization 

L 

Y 

N 

8 

87 

36d. 

Use  reference  to  correct  misspelling 

L 

Y 

Y 

20 

80 

36e. 

Apply  punctuation  rules 

L 

Y 

Y 

14 

78 

36f . 

Apply  common  grammar  rules 

L 

N 

N 

25 

80 

36g. 

Rewrite  paragraph 

M 

N 

- 

6 

67 

36h. 

Appraise  4  adjust  written  communication 

D 

Y 

- 

13 

76 

Precautions 

40a.  Use  common  knowledge  prevent  injury 

D 

Y 

10 

80 

40b. 

Minimize  safety/security  problems 

D 

Y 

- 

10 

80 

40c. 

Ident  appropriate  emergency  action 

D 

Y 

- 

12 

75 

Recoanition 

41a.  Ident  &  label  objects 

L 

Y 

Y 

8 

87 

41b. 

Use,  interpret  hand  4  arm  signals 

L 

Y 

Y 

12 

83 

41c. 

Identify  equipment  damage,  defects 

L 

Y 

N 

30 

80 

41d. 

Move,  aline,  connect  objects 

D 

Y 

- 

6 

83 

41e. 

Ident  objects  by  size,  shape,  etc. 

L 

Y 

Y 

9 

77 

4 1  f . 

No  lesson 

no  lesson 

no 

lesson 

41g. 

Choose  action  by  sight,  hearing,  touch 

D 

Y 

- 

7 

85 

41h. 

Interpret  4  use  symbols  4  codes 

L 

Y 

Y 

10 

80 

Diagnostic  Review  Lesson  (DRL)  only 
L=Both  Skill  Development  (SDL)  and 
Diagnostic  Review  (DRL)  Lessons 
M=Mixed  format 
N=No 


P=Paper  Lesson 

X=Mixed  format,  paper  segment 
not  written 
Y=Yes 
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APPENDIX  B 


Instruments 

U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences 


The  attached  data  collection  form  is  for  use  by  the  U.S.  Army  Research 
Institute  (ARI)  and  its  contractor,  The  American  Institutes  for  Research  (AIR),  in 
their  efforts  to  study  the  Job  Skills  Education  Program  (JSEP).  We  are  using  this 
form  to  survey  JSEP  instructors. 


Post- JSEP  Attitude  Survey 


Data  required  by  the  Privacy  Act  of  1974: 

PRESCRIBING  DIRECTIVE:  AR70-1 
AUTHORITY:  10  USC  SEC  4503 


PRINCIPAL  PURPOSE(S): 

The  data  collected  with  the  attached  form  are  to  be  used  for  research. 

ROUTINE  USES: 

This  is  an  experimental  personnel  data  collection  form  developed  by  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  pursuant  to  its  research  mission  as 
prescribed  in  AR  70-  When  identifiers  (name  or  Social  Security  Number)  are  requested  they 
are  to  be  used  for  administrative  and  statistical  control  purposes  only.  Full  confidentiality  of 
the  responses  will  be  maintained  in  the  processing  of  these  data. 

MANDATORY  OR  VOLUNTARY  DISCLOSURE  AND  EFFECT  ON  INDIVIDUAL 
NOT  PROVIDING  INFORMATION: 

Your  participation  in  this  research  is  strictly  voluntary.  Individuals  are  encouraged  to 
provide  complete  and  accurate  information  in  the  interests  of  the  research,  but  there  will  be 
no  effect  on  individuals  for  not  providing  all  or  any  part  of  the  information.  This  notice  may 
be  detached  from  the  rest  of  the  form  and  retained  by  the  individual  if  so  desired. 
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JSEPATTS 

Pod-JSEP  Attitude  Survey 
1/22/88 


Intryfa^oa 


The  U.S.  Army  has  asked  us  at  American  Institutes  for  Research  in  Washington,  D.C  to 
evaluate  the  Job  ran*  Education  Program  (JSEP).  Since  you  and  your  fellow  soldiers  are  the 
ones  who  have  the  most  experience  with  JSEP,  your  opinions  are  very  important  Only  you 
can  tell  us  if  JSEP  meets  your  needs,  or  how  it  can  be  improved.  Filling  out  this  question* 
naire  is  voluntary.  The  information  you  give  us  is  confidential  and  will  not  be  given  to  your 
commander  or  included  in  your  Army  record.  It  is  only  for  our  use,  so  please  be  as  frank  as 
possible. 


Directions 

Please  answer  every  question.  On  questions  where  you  can  write  comments,  feel  free  to 
write  as  much  as  you  want  Use  the  back  of  the  sheet  if  necessary.  We  want  to  know  what 
YOU  think-  After  you  have  completed  the  survey,  please  return  it  to  your  instructor.  He  or 
she  will  send  it  to  us  at  AIR.  Thank  you  very  much  for  your  help! 


pwnii  information 

Today's  Date:  / _ / _  Social  Security  Number _ - _ 

(day)  (month)  (year) 

Post _ Rank: _ Primary  MOS: _ 

L  What  MQS  are  vou  working  in  now? 

2.  How  long  have  you  been  working  in  this  MOS1  _ years  and  _ months. 

3.  How  do  you  fed  about  your  current  MOS  assignment? 

_ a)  I  like  this  job  and  want  to  keep  it 

_ b)  I  don't  like  this  job  and  want  to  change  it  because  (write  in): _ 


4.  What  is  your  native  language? 

(check  one) 

_ e)  English  _ o)  Other  (write  in) 

Participation  in  JSEP 

5.  For  how  long  did  you  take  JSEP?  _ weeks. 
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6.  Did  you  finish  ail  the  lessons  assigned  to  you  in  JSEP? 

(check  one) 

_ Y)  Yes  _ N)  No  _ S)  Not  sure 

7.  Why  did  you  enroll  in  JSEP? 

(check  all  that  apply) 

_ My  commander  recommended  or  required  it 

_ The  education  center  recommended  or  required  it 

_ I  wanted  to  take  it 

_ Other  answer  (write  in) _ _ _ 

b)  To  pass  my  SQT 
d)  To  qualify  for  a  different  MOS 
f)  To  prepare  for  the  GED  certificate 


9.  How  often  did  you  usually  participate  in  JSEP? 

_ hours  per  day  for _ days  per  week. 

Opinions  about  JSEP 

10.  Did  taking  JSEP  have  any  effect  on  your  motivation  to  learn  or 
to  improve  your  skills? 

(check  one) 

_ a)  I  became  more  motivated  while  taking  JSEP 

_ b)  I  became  less  motivated  while  taking  JESP 

_ c)  I  don't  know  what  effect  JSEP  had 

_ d)  It  had  no  effect 


8.  What  was  your  primary  purpose  in  taking  JSEP? 
(check  one) 

_ a)  To  improve  my  GT  score 

_ c)  To  improve  skills  for  my  MOS 

_ e)  To  improve  my  general  math  skills 

_ g)  To  improve  my  general  verbal  skills 

_ h)  I  didn't  know  or  wasn't  sure 

_ i)  Other  answer  (write  in) _ 
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11.  In  general,  how  difficult  were  the  JSEP  lessons? 

(check  one) 

_ a)  Too  easy 

_ b)  Rather  easy 

_ c)  Just  right 

_ d)  Rather  difficult 

_ e)  Too  difficult 

12.  What  did  you  like  about  JSEP? 

(check  all  that  apply) 

_ The  chance  to  use  the  computer 

_ The  way  JSEP  explained  things 

The  way  JSEP  drew  pictures  and  diagrams 

_ The  way  the  computer  responded  to  my  answers 

_ The  things  I  learned  in  the  lessons 

_ The  chance  to  learn  and  study  by  myself 

_ There  wasn't  anything  in  particular  that  I  liked 

_ Other  answer  (write  in) _ 

13.  What  did  you  dislike  about  JSEP? 

(check  all  that  apply) 

_ The  way  JSEP  explained  things 

_ The  way  the  computer  responded  to  my  answers 

_ The  way  JSEP  drew  pictures  and  diagrams 

_ The  way  tests  were  given 

_ Errors  in  the  JSEP  lessons 

_ Problems  with  the  computer 

_ There  wasn't  anything  that  I  really  disliked 

_ Other  answer  (write  in) _ 
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14. 


Please  rate  the  usefulness  of  different  parts  of  JSEP  lessons.  Circle  the  rating  that 
best  represents  your  opinion. 


Very  Useful 

Somewhat 

Useful 

Not 

Useful 

Don't 

Know 

The  test  before  the  lesson 

V 

S 

N 

DK 

The  short  review  lesson 

V 

S 

N 

DK 

The  test  after  the  short  lesson 

V 

S 

N 

DK 

The  long  extra-help  lesson 

V 

S 

N 

DK 

The  test  after  the  long  lesson 

V 

S 

N 

DK 

15.  What  did  JSEP  help  you  with  the  most1! 

(check  ail  that  apply) 

_ Learning  how  to  use  the  computer 

_ Learning  or  improving  skills  useful  for  my  current  job 

_ Improving  my  general  math  skills 

_ Improving  my  general  reading  and  verbal  skills 

_ Improving  my  knowledge  of  common  tasks 

_ It  didn’t  help  me  very  much  with  anything 

_ Other  answer  (write  in) _ 

16.  What  was  best  about  the  way  JSEP  lessons  work? 

(check  all  that  apply) 

_ They  explain  concepts  or  ideas  clearly 

_ They  show  how  to  do  things  step  by  step 

_ They  illustrate  things  in  practical,  useful  ways 

_ They  give  lots  of  opportunities  to  practice 

_ They  teach  things  I  never  had  the  chance  to  learn  before 

_ There  wasn't  anything  very  helpful 

_ Other  answer  (write  in) _ 
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17.  Was  there  anything  about  JSEP  that  frustrated  you  or  prevented  you  from  learning? 
(check  ail  that  apply) 

_ The  illustrations  took  too  long  to  finish 

_ The  lessons  were  boring 

_ The  lessons  repeated  things  too  much 

_ I  couldn't  work  on  the  lessons  with  other  soldiers 

_ I  didn't  make  enough  effort 

_ The  lessons  were  confusing 

_ The  lessons  were  not  the  ones  I  needed  or  wanted  to  take 

_ I  didn't  like  touching  the  screen 

_ I  couldn't  see  what  answers  I  got  wrong  on  tests 

_ I  didn't  like  how  the  computer  addressed  me 

_ The  computer  malfunctioned  too  often 

_ I  didn't  have  any  trouble  learning  from  JSEP 

_ Other  answer  (write  in) _ _ _ 


18.  What  was  the  most  difficult  part  about  JSEP  for  you? 

(check  all  that  apply) 

_ Using  the  computer 

_ Reading  the  lesson  material 

_ Understanding  the  diagrams 

_ Keeping  my  concentration 

_ The  test  questions 

_ Nothing  was  difficult 

_ Other  answer  (write  in) _ 
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19.  Please  tell  us  how  useful  the  learning  strategies  lessons  were  by  circling  the  letter  that 
best  matches  your  opinion. 


Useful 

Not  useful 

Don’t  remember 

Did  not  take 

Time  management 

U 

N 

DR 

DT 

Test  taking 

u 

N 

DR 

DT 

Motivational 

Skills 

u 

N 

DR 

DT 

Reading 

Comprehension 

u 

N 

DR 

DT 

Problem  Solving 

u 

N 

DR 

DT 

Comments  on  any  of  these  lessons? 


20.  Did  you  have  any  computer  problems  while  using  JSEP? 

(check  ail  that  apply) 

_ The  computer  screen  became  messed  up 

_ I  got  "stuck"  in  a  lesson 

_ The  computer  marked  my  correct  answer  wrong 

_ The  computer  did  not  respond  to  my  commands 

_ The  computer  was  broken 

_ There  were  not  enough  computers 

_ I  didn’t  have  any  problems 

2L  What  do  you  think  about  the  lessons  assigned  to  you  in  JSEP? 
(check  all  that  apply) 

_ The  lessons  were  too  easy 

_ They  were  just  the  ones  I  needed  to  take 

_ I  wanted  to  change  the  lessons  assigned  to  me  but  couldn’t 

_ Most  of  the  lessons  were  not  the  ones  I  wanted  or  needed 

_ Other  answer  (write  in) _ 
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22.  How  do  you  usually  Ieam  best— by  using  the  computer  or 
from  a  teacher  in  a  classroom? 

(check  one) 

_ a)  It  makes  no  difference 

_ b)  From  the  computer,  in  general 

_ c)  From  the  teacher,  in  general 

_ d)  It  depends-sometimes  I  prefer 

the  computer  and  sometimes 
I  prefer  the  teacher 

_ e)  Not  sure 

23.  Here  are  some  things  that  usually  help  adults  Ieam  and  remember  new  skills.  For  each 
one,  please  circle  the  letter  that  shows  whether  JSEP  helped  you  in  this  way. 


Did  JSEP  h?|p  ygu: 

Yes 

No 

Don't  know 

Learn  independently  at  your  own  pace? 

Y 

N 

DK 

Practice  something  over  and  over 
again  if  you  didn't  get  it 
right  the  first  time? 

Y 

N 

DK 

Be  motivated  to  Ieam? 

Y 

N 

DK 

Relax  and  not  worry  about  your  mistakes? 

Y 

N 

DK 

Discover  how  to  do  something  yourself 
instead  of  being  told  how  to  do  it? 

Y 

N 

DK 

See  material  presented  clearly  step  by 
step? 

Y 

N 

DK 

24.  How  often  did  you  go  to  the  instructor  for  help 
while  you  were  working  on  JSEP? 

(check  one) 

_ a)  Usually  more  than  three  times  per  session 

_ b)  Between  one  and  three  times  per  session 

_ c)  About  once  per  session 

_ d)  Less  than  once  per  session 

_ e)  Never  or  rarely 
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25.  What  kinds  of  problems  did  you  usually  ask  your  instructor  about? 
(check  all  that  apply) 

_ Problems  with  the  computer 

_ Problems  understanding  parts  of  the  lesson 

_ Problems  with  practice  questions 

_ Problems  with  tests 

_ I  rareiy  or  never  asked  about  problems 

_ Other  answer  (write  in) _ _ 


26.  How  did  your  instructor  usually  help  you? 

(check  all  that  apply) 

_ Told  me  to  keep  trying  or  to  review  the  lesson  again 

_ Tried  to  explain  something  in  a  different  way 

_ Tried  tc  adjust  or  fix  the  computer 

_ Gave  me  other  books  or  materials  to  read 

_ Did  not  or  could  not  help  me 

_ I  rarely  or  never  asked  the  instructor  for  help 

_ Other  answer  (write  in) _ 


27.  How  much  do  you  think  you  learned  from  JSEP? 
(check  one) 

_ a)  I  learned  a  lot 

_ b)  I  learned  some 

_ c)  I  didn't  learn  very  much 

_ d)  I  didn't  learn  anything 

_ e)  Not  sure  or  don't  know 
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23.  Please  rate  the  effect  of  taking  JSEP  on  how  well  you  do  your  job. 

(check  one) 

_ a)  Positive  effect-I’ve  learned  many  useful  things  for  my  job 

_ b)  Average  effect-It  has  helped  me  a  little  bit  on  my  job 

_ c)  Negative  effect-It  has  had  a  bad  effect  on  my  job  performance 

_ d)  No  effect  on  my  job  performance 

_ e)  Not  sure  or  don’t  know 

29.  How  would  you  recommend  JSEP  be  used? 

(check  one) 

a)  Use  JSEP  as  a  replacement  for  all  BSEP  classroom  instruction 

_ b)  Use  JSEP  as  part  of  BSEP  classroom  instruction 

_ c)  Don't  use  JSEP  at  all 

_ d)  Not  sure  or  don’t  know 

_ e)  Other  (Write  in) 

30.  How  did  you  first  feel  about  JSEP  when  you  started  it? 

(check  one) 

_ a)  I  liked  it  a  lot 

_ b)  I  liked  it  a  little 

_ c)  I  didn't  like  it  very  much 

_ d)  I  didn’t  like  it  at  all 

31.  How  do  you  feel  about  JSEP  now? 

(check  one) 

_ a)  I  like  it  a  lot 

_ b)  I  like  it  a  little 

_ c)  I  don't  like  it  very  much 

_ d)  I  don't  like  it  at  all 
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32.  Was  there  anything  that  made  your  feelings  change? 


N)  No _ Y)  Yes.  What?. 


33.  In  what  ways  has  taking  JSEP  helped  you  or  caused  you  problems? 
(check  all  that  apply) 

_ JSEP  taught  me  what  I  need  to  know  for  my  job 

_ JSEP  increased  my  self-confidence 

_ My  commander  didn't  like  me  spending  time  in  JSEP 

_ JSEP  motivated  me  to  learn  more  and  improve  my  skills 

_ JSEP  taught  me  a  lot  about  the  Army  in  general 

_ JSEP  held  me  back  from  my  unit 

_ JSEP  helped  me  to  improve  my  GT  score 

_ JSEP  helped  me  to  learn  a  new  MOS 

_ JSEP  gave  me  a  break  from  the  regular  routine 

_ There  were  no  major  benefits  from  JSEP 

_ There  were  no  major  problems  from  JSEP 

_ Other  answer  (write  in) _ _ 


34.  Would  you  like  to  change  anything  in  the  way  JSEP  is  set  up? 

_ N)  No,  I  wouldn't  change  anything 

_ Y)  Yes,  I  would  change  (write  in  answer) _ 
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35.  Please  rate  the  following  JSEP  elements.  Circle  the  letter  which  best  represents  your 
opinion. 

Poor. 

Needs  some  needs  a  lot  of 

Excellent  Good  improvement  imorovemenr 

The  pictures  on  the  screen 

E 

G 

I 

P 

The  explanations  of  basic 
ideas 

E 

G 

I 

P 

The  examples  and  practice 
questions 

E 

G 

I 

P 

The  computer  responses  to 
answers  on  practice 
questions 

E 

G 

I 

P 

The  level  of  difficulty 
of  the  reading  in  the 
lessons 

E 

G 

I 

P 

The  way  tests  are  scored 

E 

G 

I 

P 

The  number  of  tests  required 

E 

G 

I 

P 

The  computer  set-up 

E 

G 

I 

P 

Program  features  letting  you 
control  the  computer  screen 
or  move  through  the  lesson(s) 

E 

G 

I 

P 

The  choice  of  lessons  you  had 
to  take 

E 

G 

I 

P 

The  length  of  the  lessons 
you  took 

E 

G 

I 

P 

The  amount  of  time  you  spent 
in  JSEP 

E 

G 

I 

P 

The  number  of  lessons  assigned 
to  you 

E 

G 

I 

P 

Help  from  the  instructor 

E 

G 

I 

P 

The  military  content  of  the 
lessons 

E 

G 

I 

P 

The  civilian  content  of  the 
lessons 

E 

G 

I 

P 

— 
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Comments.  Please  explain  any  rating  you  gave  in  the  last  question  that  could  help  improve 
JSEP  for  other  'ioidiers. 


36.  The  next  10  questions  ask  for  a  summary  of  your  attitudes  about  JSEP.  Please  indicate 
how  much  you  agree  with  each  statement  by  circling  the  number  that  best  reflects  your 
opinion: 


A.  It  was  easy  for  me  to  learn  to  use  the  computer  1  2  3  4  5 


B.  Using  the  computer  for  three  or  four  hours  at  a  time  was 
too  long 

1 

2 

3 

4 

5 

C  I  think  written  assignments  should  be  used  along 
with  the  computer 

1 

2 

3 

4 

5 

D.  I  think  the  lessons  will  help  me  to  read  and 
understand  the  publications  I  use 

I 

2 

3 

4 

5 

E  The  skills  I’ve  learned  in  JSEP  will  help  me  to 

advance  to  a  higher  grade/rank  in  the  Army 

1 

2 

3 

4 

5 

F.  I  think  the  instructor  in  the  Education  Center  should 
teach  the  lessons  instead  of  the  computer 

1 

2 

3 

4 

5 

G.  I  would  be  willing  to  take  more  JSEP  lessons  on  a  computer 
if  thev  were  offered  durinz  on-dutv  hours 

1 

2 

3 

4 

5 

H.  I  would  be  willing  to  take  more  JSEP  lessons 
offered  on  a  computer  if  they  were  offered  during 
off-dutv  hours 

1 

2 

3 

4 

5 

L  I  think  my  unit  commander  would  be  willing  to 
release  me  from  duty  to  take  JSEP  lessons 

1 

2 

3 

4 

5 

J.  I  think  JSEP  should  be  included  as  an  educational 
program  offered  by  the  Education  Center 

1 

2 

3 

4 

5 
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Paper-Based  Lessonst  Please  record  the  Lesson  Humber  (LS#)  and  Percent  Scored  Correct  (%SC)  on 

the  lesson  test  for  any  paper-based  lessons  completed  by  each  soldier. 
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Prepared  by:  American  Institutes  for  Research 
1055  Thomas  Jefferson  Street,  N. 


U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences 


The  attached  data  collection  form  is  for  use  by  the  U.S.  Army  Research 
Institute  (AJRI)  and  its  contractor,  The  American  Institutes  for  Research  (AIR),  in 
their  efforts  to  study  the  Job  Skills  Education  Program  (JSEP).  We  are  using  this 
form  to  survey  JSEP  instructors. 


JSEP  Instructor  Survey 


Data  required  by  the  Privacy  Act  of  1974: 

PRESCRIBING  DIRECTIVE:  AR  70-1 
AUTHORITY:  10  USC  SEC  4503 


PRINCIPAL  PURPOSE(S): 

The  data  collected  with  the  attached  form  are  to  be  used  for  research. 

ROUTINE  USES: 

This  is  an  experimental  personnel  data  collection  form  developed  by  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  pursuant  to  its  research  mission  as 
prescribed  in  AR  70-1.  When  identifiers  (name  or  Social  Security  Number)  are  requested  they 
are  to  be  used  for  administrative  and  statistical  control  purposes  only.  Full  confidentiality  of 
the  responses  will  be  maintained  in  the  processing  of  these  data. 

MANDATORY  OR  VOLUNTARY  DISCLOSURE  AND  EFFECT  ON  INDIVIDUAL 
NOT  PROVIDING  INFORMATION: 

Your  participation  in  this  research  is  strictly  voluntary.  Individuals  are  encouraged  to 
provide  complete  and  accurate  information  in  the  interests  of  the  research,  but  there  will  be 
no  effect  on  individuals  for  not  providing  all  or  any  part  of  the  information.  This  notice  may 
be  detached  from  the  rest  of  the  form  and  retained  by  the  individual  if  so  desired. 
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JSEP-IS 

JSEP  Instructor  Survey 


Instructions 

The  Department  of  the  Army  has  asked  us  at  the  American  Institutes  for  Research  in 
Washington,  D.C  to  survey  you  about  your  preparation  as  an  instructor  for  JSEP,  and  your 
evaluation  of  how  JSEP  is  working. 

It  will  not  take  long  to  fill  out  this  survey.  You  can  answer  most  of  the  questions  by 
checking  the  responses  that  best  fit  your  experience.  Filling  out  this  survey  is  voluntary.  Your 
answers  will  help  us  to  find  out  more  about  JSEP.  Your  comments  will  not  be  identified  in  any 
reports.  Please  return  the  completed  survey  to  AIR  in  the  preaddressed  envelope  provided  for  the 
return  of  evaluation  materials.  Thank  you  very  much  for  helping  us. 


1.  Today’s  date  /  / 

(day  /  month  /  year) 

2.  What  is  the  highest  degree  that  you  have 
completed? 


5.  Have  you  taught  BSEP  or  JSEP  before? 
_  No 

_  Yes 

If  you  answered  yes,  for  how  long? 
_ years  and _ months 

6.  For  how  long  have  you  been  teaching  with 
JSEP? 

_ years  and _ months 


Using  computers  in  general 
Teaching  enlisted  military  personnel 


3.  Please  check  any  of  the  following  that  you 
have  studied: 

_  Adult  education 

_  Educational  foundations  (e.g^  learning 

theory,  philosophy,  etc.) 

_  Teaching  methodology 

_ Special  education 

4.  Before  you  were  hired  as  a  JSEP  instructor, 
in  which  of  the  following  did  you  have 
experience? 

_ Teaching  adult  basic  education/GED 

_ Teaching  with  computer-based 

instructional  materials 
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Co  co 


EP-IS 

EP  Instructor  Survey 


Who  trained  you  to  become  a  JSEP  instructor,  and  how  long  was  your  training?  (If  you  did 
not  receive  any  formal  JSEP  training,  write  "0"  on  the  line  next  to  "I  did  not  receive  any 
formal  training.") 

Length  of  time  for  training 

T  was  trained  bv:  (hours  /  days  /  weeks’): 

An  FSU  staff  member . . 

An  ACES  staff  member . . 

I  did  not  receive  any  formal  training. 

I  learned  JSEP  "on  the  job." . . 

Other  answer  (write  in)  _  .  •  . _ 


8.  Please  rate  how  important  you  feel  it  is  for  new  JSEP  instructors  to  receive  training  in  these 
areas,  and  then  indicate  whether  these  areas  were  covered  in  your  JSEP  training. 

•  Under  "Important?"  circle  the  rating  that  best  reflects  your  opinion. 

•  Under  "Covered?"  circle  the  rating  that  reflects  whether  the  area  was  covered  in  your  JSEP 
training.  If  you  did  not  receive  any  formal  JSEP  training,  circle  "N." 

Important?  Covered? 


Somewhat  Not 


Training  new  instructors  in: 

Important 

important 

Important 

Yes 

Operating  the  computer 

IM 

SI 

NI 

Y 

Getting  in  and  out  of  JSEP 

IM 

SI 

NI 

Y 

Registering  a  soldier 

IM 

SI 

NI 

Y 

Dealing  with  computer  problems 

IM 

SI 

NI 

Y 

Helping  soldiers  with  learning 
difficulties 

IM 

SI 

NI 

Y 

Training  other  instructors 

IM 

SI 

NI 

Y 

What  to  do  if  soldiers  get  stuck 
in  a  lesson 

IM 

SI 

NI 

Y 

Managing  the  JSEP  classroom 

IM 

SI 

NI 

Y 

Operating  the  learning 
strategies  modules 

IM 

SI 

NI 

Y 
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JSEP  -  IS 

JSEP  Instructor  Survey 


9.  Please  list  additional  areas,  if  any,  that  should  be  covered  in  the  training  for  new  JSEP 
instructors. 


10.  Please  rate  how  well  you  were  prepared  in  these  areas.  Circle  the  rating  that  best  represents 
your  opinion.  If  the  area  is  not  relevant  or  requires  no  training,  circle  the  rating  under  "Does 
not  apply." 


Very  good 
preparation 

Fair 

Preparation 

Poor/No 

Preparation 

Does  n 
applv 

Operating  the  computer 

VG 

F 

P 

NA 

Getting  in  and  out  of  JSEP 

VG 

F 

P 

NA 

Registering  a  soldier 

VG 

F 

P 

NA 

Dealing  with  computer  problems 

VG 

F 

P 

NA 

Helping  soldiers  with  learning 
difficulties 

VG 

F 

P 

NA 

Training  other  instructors 

VG 

F 

P 

NA 

What  to  do  if  soldiers  get  stuck 
in  a  lesson 

VG 

F 

P 

NA 

Managing  the  JSEP  classroom 

VG 

F 

P 

NA 

Operating  the  learning  strategies 
modules 

VG 

F 

P 

NA 

11.  Please  list  the  areas,  if  any,  in  which  experienced  JSEP  instructors  might  need  additional 
training. 
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JSEP  -  IS 

JSEP  Instructor  Survey 


After  you  began  working  as  a  JSEP  instructor,  did  you  have  any  of  the  following  resources  to 
help  you  get  more  information  about  JSEP?  Please  circle  the  rating  that  reflects  whether  you 
had  the  resource,  and  whether  it  was  helpful. 

Had? 

Heloful? 

Resource: 

Yes 

No 

Yes 

No 

Contact  with  JSEP  resource  people  by  telephone . 

.  .  .  .  Y 

N 

Y 

N 

Contact  with  JSEP  resource  people  through  computer  connections  .  .  . 

.  .  .  .  Y 

N 

Y 

N 

Training  workshops . 

.  ...  Y 

N 

Y 

N 

Meetings  or  conferences . 

The  JSEP  Instructor's  Manual,  the  JSEP  Soldier  Management 

.  ...  Y 

N 

Y 

N 

System:  Features  and  Functions,  and  other  support  materials . 

.  ...  Y 

N 

Y 

N 

Other  answer  (write  in) 

Y 

N 

Y 

N 

13.  What,  if  any,  additional  support  people,  hardware,  or  systems  for  the  operation  of  JSEP  will 
JSEP  instructors  need  in  the  future? 


14. 


In  general,  as  a  JSEP  instructor,  how  much  time  do  you  spend  doing  the  following  activities? 
Estimate  the  percentage  of  time  you  spend  on  each  task.  Remember  that  your  answers  should 
sum  to  100%. 

Showing  soldiers  how  to  use  the  program  and  hardware . _ 

Circulating  and  observing  while  soldiers  use  JSEP. . _ 

Helping  to  solve  computer  problems  with  the  program  or  hardware . _ 

Keeping  records  and  doing  other  administrative  tasks . _ 

Using  supplemental  instructional  materials  with  soldiers . _ 

Answering  soldiers’  questions  about  the  lessons . _ 

Other  activity  (write  in)  _  .  .  _ 

Total: . 


% 


% 


% 


100% 
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JSEP -IS 

JSEP  Instructor  Survey 

15. 

Please  rate  the  following  aspects  of  JSEP.  Circle  the  rating  that  best  represents  your  opinion. 

Excellent 

Needs  some 
Good  imDrovement 

Poor,  needs 
a  lot  of 
imorovement 

Can’t 

Judse 

The  on-line  instructional  program 
(the  JSEP  lessons) 

E 

G 

0 

P 

D 

The  paper-based  JSEP  lessons 

E 

G 

0 

P 

D 

The  soldier  management  system 

E 

G 

0 

P 

D 

The  hardware  (computers  and  related 
equipment) 

E 

G 

0 

P 

D 

The  classroom  setting  (i.e.,  the  classroom, 
the  lighting,  how  the  computers 
are  set  up,  etc) 

E 

G 

0 

P 

D 

The  JSEP  Instructor's  Manual  &  the  JSEP 
Soldier  Management  System:  Features  and 
Functions) 

E 

G 

o 

P 

D 

The  JSEP  training 

E 

G 

0 

P 

D 

16. 

How  do  you  feel  that  JSEP  can  be  used  best? 

Use  JSEP  as  a  replacement  for  all  BSEP  classroom  instruction 

Use  JSEP  as  part  of  BSEP  classroom  instruction 

Don’t  use  JSEP  at  all 

Not  sure  or  don't  know 

Other  (write  in) 

17. 

Please  give  us  any  other  comments  about  JSEP  that  you  would  like  to  add. 
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JSEP  INTERVIEW  PROTOCOLS 
Educational  Staff  Questions 


Introduction:  Purpose  is  to  provide  information  that  will  assist 
the  Army  in  using  JSEP  as  effectively  and  efficiently  as 
possible.  Interviews  best  way  to  get  judgments  and  insights  of 
those  who  have  experience  with  JSEP  and  to  whom  the  program  is 
important. 

o  Voluntary  participation  in  interview 
o  No  AIR  report,  formal  or  informal,  will  identify 
respondents 

Interviewer:  Ask  if  there  are  any  questions.  Ask  if  respondent 
agrees  to  interview.  Ask  major  question;  follow  up  with  probe 
question  if  respondent  is  not  forthcoming  on  that  point. 


Respondent: _  Post: _ 

Title:  _  _  Date: _  Int: 


1.  What  has  been  your  role  or  background  in  JSEP? 

Probe:  What  do  you  do  with  JSEP?  How  long  have  you  worked 
with  JSEP?  Prior  experience  with  related/similar  programs? 

2.  From  this  perspective,  I  would  like  you  to  give  your 
evaluation  of  JSEP  in  several  areas. 

a)  How  well  do  you  think  JSEP  is  meeting  its  purposes? 

Probe:  What  purposes  does  it  address?  What  purposes  should 
it  address? 

b)  How  does  JSEP  fit  in  or  coordinate  with  other  Army  education? 

Training?  General  operations  on  the  post? 

Probe:  Can  you  give  some  examples  that  illustrate  that? 

c)  What  are  JSEP's  strengths? 

Probe:  Best  points?  Things  it  does  that  prior  programs 
didn't?  Any  unexpected  benefits? 

d)  What  are  JSEP's  weaknesses? 

Probe:  Things  it  needs  to  do  but  doesn't?  Areas  that  need 
revision,  deletion? 

3.  JSEP  is  at  the  point  of  changing  from  a  pilot  project  under 
development  by  Florida  State  University  to  a  regular  education 
program  managed  completely  by  the  Army.  From  your  experience 
what  advice  can  you  offer  about  the  following  management  areas? 

Probe:  for  each,  Is  it  adequate  now?  What  needs  to  be  done? 
By  whom? 

a)  Training  for  new  instructors,  in-service  for  experienced  ones? 

b)  Material  resources  such  as  Instructor's  Manuals,  computer 

hardware,  programming  to  diagnose  soldiers'  needs, 
select  lessons,  report  progress?  Consumable  materials? 

c)  Keeping  curriculum  updated,  lessons  well-written? 

dj  What  do  you  think  JSEP  will  be  used  for,  or  how  will  it  be 
used,  two  years  from  now? 
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4.  Anything  you  would  like  to  add,  any  questions? 


Thanks ! 


Questions  for  Training,  Duty  Supervisors 
(Introduction  same  as  for  Educational  Staff.) 

Respondent: _  Post: _ 

Title: _  Date:  Int: 


1.  How  familiar  are  you  with  JSEP?  How  has  it  affected  your  area 
of  responsibility? 

Probe:  Refer  soldiers?  Receive  JSEP  "graduates?"  General 
working  relationship  with  the  Education  Center? 

2.  From  your  perspective,  how  well  do  you  think  JSEP  is  working? 

Probe:  For  your  needs,  what  should  it  do?  What  does  it  do? 
How  important  are  the  things  JSEP  is  designed  to  do? 

3.  What  do  you  see  as  JSEP's  strengths? 

Probe:  What  it  does  well?  Expected  and  unexpected  pluses? 
Compared  to  earlier  programs? 

4.  What  do  you  see  as  JSEP's  weaknesses? 

Probe:  Areas  to  be  added  or  deleted?  Problems  it  causes 
unintentionally? 

5.  Anything  you  would  like  to  add,  or  ask  about?  Thanks! 


Questions  for  Soldiers 
(Introduction  same  as  previous) 


Respondent: _  Post: _ 

Rank:  _  _  Date:  _  Int: 


1.  How  long  have  you  been  working  with  JSEP?  What  do  you  want 
to  get  out  of  it  (SQT  score,  MOS  improvement,  etc.)?  Have  you 
had  any  other  Army  educational  programs,  like  BSEP? 

2.  From  your  experience,  how  does  JSEP  compare  with  other 
classes  or  study  programs  you  have  taken?  Do  you  like  working 
with  JSEP?  Is  it  a  good  way  for  you  to  learn? 


3.  Have  you  been  able  to  accomplish  your  learning  goals  through 
JSEP —  or  do  you  think  you  will  by  the  time  you  finish?  Is  what 
you  are  learning  relevant  to  your  job? 

4.  What  are  JSEP's  strong  points?  What  is  most  effective,  what 
do  you  like  the  most? 

5.  What  are  JSEP's  weaknesses?  Things  that  need  to  be  changed, 
added,  dropped? 

6.  Have  there  been  any  benefits  to  you  from  participating  in 
JSEP?  Has  it  caused  you  any  problems?  Give  some  examples  to 
illustrate. 

7.  What  advice  would  you  give  the  Army  about  using  JSEP  with 
other  soldiers  like  yourself? 

8.  Anything  you  would  like  to  add,  or  ask  about?  Thanks! 
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APPENDIX  C 


Specific  Lesson-Level  Performance  and  Time  Data 

Appendix  C  JSEP  Lesson  Level  Results 

The  purpose  of  this  section  is  to  present  detailed  information  about  the  performance 
of  soldiers  on  JSEP  on-line  lessons,  in  terms  of  mastery  rates  and  the  amount  of  time 
used  to  complete  the  lessons.  It  is  appended  to  the  technical  report  because  the 
information  is  expected  to  be  of  interest  to  a  limited  group  of  readers. 

Lesson  Mastery  Rates 

Table  C.1  shows  the  number  of  soldiers  failing  the  pretest  for  each  lesson,  which  is 
also  the  number  going  through  the  lesson’s  instructional  content  Of  that  group,  the  table 
shows  the  percent  passing  the  short  or  long  lesson  test,  or  failing  the  lesson.  For 
example,  30  percent  of  those  attempting  lesson  la  failed  the  pretest  Of  these  53 
soldiers,  66  percent  passed  after  taking  the  short  lesson,  27  percent  after  the  long  lesson, 
and  8  percent  did  not  pass  the  lesson.  Referring  to  Table  C.1,  the  following  sections 
discuss  soldier  performance  for  each  series,  beginning  with  mathematics. 

Math  lesson  performance.  The  lessons  in  the  Numbering  and  Counting  Series  did  not 
appear  to  be  as  easy  as  their  names  suggested.  Only  lessons  la  and  le  were  passed  on 
the  pretest  by  half  or  more  of  the  soldiers  attempting  them.  Of  soldiers  who  went 
through  the  lessons,  49  percent  failed  lesson  lc  and  64  percent  failed  lesson  li. 

In  the  lessons  on  linear,  Weight  and  Volume  Measures,  the  pretests  were  failed  for 
2b  and  2g  more  frequently  than  for  other  lessons.  Soldiers  were  most  likely  to  fail  lesson 
2g  (33%)  or  2e  (21%).  Virtually  none  of  the  soldiers  passed  the  pretest  for  lesson  3c 
(azimuths  and  mils)  in  the  series  on  Degree  Measures.  However,  75  percent  or  more  of 
the  soldiers  taking  the  lessons  in  this  series  passed  them. 

Soldiers  were  largely  unsuccessful  on  the  pretests  for  the  Time-Telling  Measures 
Series,  except  for  the  lesson  on  estimating  seconds,  minutes  and  parts  of  an  hour.  One- 
third  of  the  soldiers  failed  the  lesson  on  equating  Gregorian  and  Julian  dates,  while  two 
of  the  three  soldiers  who  took  the  lesson  on  converting  to  Zulu  time  failed  it. 
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Table  C.l.  Soldiers  Failing  Pretest:  Percent  Passing  or  Failing 
(On-Line  Lessons  Only)  (N=179) 


Lesson 

Of  Attempting: 
Fail  Pretest 
(No.)  (Pet.) 

Of  Failing:* 
Pass  Short 
(Pet.) 

Pass  Long 
(Pet.) 

Not  Pass 
(Pet.  L- 

Numberina  &  Countina 
la.  Match  name 

53 

30.3 

66.0 

26.4 

7.5 

lb.  Sequence 

113 

64.9 

36.3 

37.2 

26.5 

lc.  Before,  between 

169 

96.6 

23.7 

27.2 

49.1 

Id.  Greater/lesser 

91 

52.3 

34.1 

37.4 

28.6 

le.  Ordinal  position 

39 

22.4 

82.1 

12.8 

5.1 

If.  Place  value 

89 

50.9 

92.1 

3.4 

4.5 

lg.  Round  whole/decimal 

127 

73.0 

57.5 

26.8 

15.7 

lh.  Count  by  1,5,10,  etc. 

121 

70.8 

64.5 

-- 

35.5 

li.  Match  scale  intervals 

122 

70.9 

36.1 

-- 

63.9 

Linear,  Weight  &  Volume 
Measures 


2a. 

Linear  scale  mark 

26 

15.6 

57.7 

38.5 

3.8 

2b. 

US  &  metric  measures 

87 

51.5 

88.5 

11.5 

0.0 

2c. 

Lengths 

24 

14.4 

91.7 

8.3 

0.0 

2d. 

Weight,  pressure,  torque 

6 

11.3 

83.3 

-- 

16.7 

2e. 

Volume 

67 

39.6 

79.1 

20.9 

2f . 

Non-numeric  calibrated 

65 

38.9 

92.3 

7.7 

2g. 

Estimate  size,  distance 

132 

79.0 

53.8 

12.9 

33.3 

Deqree  Measures 

3a. 

Degrees,  mils,  angles, 

temp 

59 

36,0 

84.7 

13.6 

1.7 

3b. 

Est  angle  <180  deg 

61 

37.2 

67.2 

18.0 

14.8 

3c. 

Interp  azimuths,  0-6400 

mils 

153 

94.4 

37.5 

37.9 

24.2 

3d. 

Interp  azimuths,  0-360 

deg 

76 

46.9 

73.7 

15.8 

10.5 

Time- 

-Tell ina  Measures 

4a. 

Digital ,  analog,  24  hr 

150 

88.8 

68.7 

24.7 

6.7 

4b. 

Clockface  direction 

84 

50.0 

91.7 

8.3 

0.0 

4c. 

Est  secs,  mins 

32 

19.0 

75.0 

9.4 

15.6 

4d. 

Gregorian  &  Julian 

168 

98.8 

18.5 

48.8 

32.7 

4e. 

Convert  hours,  lOths 

45 

100.0 

71.1 

28.9 

4f . 

Convert  to  Zulu 

3 

100.0 

0.0 

33.3 

66.7 

*Percents  based  upon  number  failing  pretest. 
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Table  C.l.  (Cont.)  Soldiers  Failing  Pretest:  Percent  Passing  or  Failing 

(On-Line  Lessons  Only)  (N=179) 


Lesson 

Of  Attempting: 
Fail  Pretest 
(Ho.)  (Pet.) 

Of  Fail ing:* 
Pass  Short 
_ (Pet.) _ 

Pass  Long 
_ LPct.I _ 

Not  Pass 
(Pet.)  _ 

Gaqe  Measures 

5a.  Read  gage 

134 

82.7 

84.3 

13.4 

2.2 

5b.  Read  read-out 

48 

29.6 

93.8 

4.2 

2.1 

5c.  Read  gage,  color 

30 

18.5 

96.7 

3.3 

0.0 

5d.  Read  scales  (+/-) 

4 

2.5 

75.0 

25.0 

0.0 

5e.  Read  multiscale 

59 

36.4 

50.8 

44.1 

5.1 

5f.  Match  gage  to  spec 

40 

24.8 

67.5 

17.5 

15.0 

5g.  Read  unnumbered/unmarked 

1  6 

3.7 

66.7 

33.3 

0.0 

5h.  Read  fluctuating  gage 

76 

47.2 

75.0 

13.2 

11.8 

5i.  Match  specs/aline 

15 

9.3 

100.0 

0.0 

0.0 

Spatial 

6a.  Ident  direct  tools  move 

2 

1.3 

100.0 

0.0 

0.0 

6b.  Manip  to  aline,  etc. 

102 

66.2 

60.8 

22.5 

16.7 

6c.  2-dimension  to  spatial 

7 

4.5 

71.4 

28.6 

0.0 

6d.  Symbols  to  systems 

81 

52.6 

60.5 

-- 

39.5 

Lines 

7a.  Ident  points,  etc. 

88 

56.1 

94.3 

5.7 

0.0 

7b.  Ident  vert,  horiz,  diag 

66 

42.0 

81.8 

13.6 

4.5 

7c.  Ident  intersect,  diverg 

64 

40.8 

90.6 

7.8 

1.6 

7d.  Superimpose  lines 

6 

100.0 

100.0 

-- 

0.0 

PI anes 

8a.  Match  plane  shapes 

31 

19.9 

100.0 

0.0 

0.0 

8b.  Ident  geometric 

0 

0.0 

0.0 

-- 

0.0 

8c.  Apply  shape  terms 

65 

41.7 

55.4 

21.5 

23.1 

8d.  Match  patterns 

29 

18.6 

58.6 

20.7 

20.7 

8e.  Figure  orient 

15 

9.6 

80.0 

-- 

20.0 

Anales  4  Tri«(nql?$ 

9a.  Ident  angles 

136 

84.0 

70.6 

20.6 

8.8 

9b.  Ident  types  angles 

17 

94.4 

82.4 

-- 

17.6 

9c.  Ident  types  triangles 

5 

55.6 

100.0 

0.0 

0.0 

9d.  Ident  altitudes,  bisect 

122 

75.3 

69.7 

27.9 

2.5 

9e.  Name  angles 

10 

58.8 

100.0 

-- 

0.0 

Sol  ids 

10a.  Match  names/figures 

139 

82.7 

75.5 

22.3 

2.2 

♦Percents  based  upon  number  failing  pretest. 
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Table  C.l.  (Cont.)  Soldiers  Failing  Pretest:  Percent  Passing  or  Failing 

(On-Line  Lessons  Only)  (N-179) 


Lesson 

Of  Attempting: 
Fail  Pretest 
(No.)  (Pet.) 

Of  Failing:* 
Pass  Short 
(Pet.) 

Pass  Long 
(Pet.) 

Not  Pass 
(Pet. ) 

Addition  &  Subtraction 

12a.  Whole,  no  carry 

1 

0.6 

100.0 

0.0 

0.0 

12b. 

Whole,  carry 

17 

10.2 

88.2 

5.9 

5.9 

12c. 

Decimals,  carry 

31 

18.6 

74.2 

22.6 

3.2 

1 2d . 

Pos  &  neg  numbers 

142 

86.6 

47.9 

38.0 

14.1 

12e. 

24-hour  time 

80 

48.8 

47.5 

27.5 

25.0 

12f . 

Increments  measuring 

43 

26.5 

72.1 

23.3 

4.7 

12g. 

Linear,  dry,  liquid,  deg 

49 

84.5 

53.1 

26.5 

20.4 

12h. 

Estimate  sum,  diff 

160 

98.8 

18.1 

22.5 

59.4 

MultiDl ication  &  Division 

13a.  Whole  numbers 

25 

16.3 

80.0 

20.0 

0.0 

13b. 

Whole  &  decimal 

85 

55.2 

56.5 

29.4 

14.1 

13c. 

Deci ,  divisor,  dividend 

104 

68.9 

29.8 

13.5 

56.7 

13d. 

Neg  &  pos  numbers 

17 

70.8 

100.0 

0.0 

0.0 

13e. 

Est  product,  quotient 

5 

25.0 

80.0 

0.0 

20.0 

Fractions/Decimal s 

14a.  Est  frac  length,  area, 
vol 

49 

32.9 

55.1 

22.4 

22.4 

14b. 

Reduce 

35 

23.5 

77.1 

11.4 

11.4 

14c. 

Convert  decs/fracs 

8 

40.0 

50.0 

12.5 

37.5 

14d. 

Convert  decimal  s/%' s 

88 

59.5 

89.8 

6.8 

3.4 

14e. 

Add,  subtract  fracs 

27 

48.2 

81.5 

14.8 

3.7 

14f . 

Multiply,  divide  fracs 

107 

72.3 

68.2 

27.1 

4.7 

14g. 

Estimate  fraction 

27 

77.1 

25.9 

33.3 

40.7 

Geometry 

15a.  Draw  plane  fig 

23 

15.8 

100.0 

0.0 

15b. 

Match  figures/names 

24 

16.4 

87.5 

8.3 

4.2 

15c. 

Label  objects,  figures 

20 

13.7 

100.0 

-- 

0.0 

15d. 

Use  protractor 

25 

17.5 

24.0 

44.0 

32.0 

15f . 

Area,  perimeter  rectang 

36 

97.3 

61.1 

38.9 

0.0 

15g. 

Radius,  area,  circum  cir 

25 

89.3 

92.0 

4.0 

4.0 

15h. 

Measure  rectang  solids 

35 

100.0 

51.4 

31.4 

17.1 

1 5  i . 

Geometric  prob  formulas 

8 

50.0 

87.5 

12.5 

0.0 

15  j . 

Oscilloscope  readouts 

11 

100.0 

9.1 

36.4 

54.5 

*Percents  based  upon  number  failing  pretest. 
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Table  C.l.  (Cont.)  Soldiers  Failing  Pretest:  Percent  Passing  or  Failing 

(On-Line  Lessons  Only)  (N-179) 


Lesson 

Of  Attempting: 
Fail  Pretest 
(Ho.)  f Pet . 1 

Of  Failing:* 
Pass  Short 
(Pet.) 

Pass  Long 
(Pet.) 

Not  Pass 
(Pet.) 

Combination  of  Processes 

16a.  Locate  center  of  object 

0 

0.0 

0.0 

0.0 

16b. 

Compute  averages 

51 

35.4 

74.5 

9.8 

15.7 

16c. 

All  proc,  whole/mixed 

19 

82.6 

31.6 

31.6 

36.8 

16d. 

All  proc,  units  meas 

20 

95.2 

20.0 

35.0 

45.0 

16e. 

Info  charts,  graphs,  etc 

:.  5 

45.5 

80.0 

20.0 

0.0 

16f . 

Conversion  probs 

29 

70.7 

75.9 

10.3 

13.8 

16g. 

Ratio,  proportion  probs 

98 

68.5 

71.4 

10.2 

18.4 

16h. 

All  proc,  word  probs 

12 

50.0 

66.7 

25.0 

8.3 

Graohina  in  the  Coordinate 
PI  ane 


17a. 

Grid  coords  map 

80 

53.0 

76.3 

20.0 

3.8 

17b. 

6-digit  coords 

33 

21.9 

78.8 

-- 

21.2 

17c. 

Plot  point,  dist,  dir 

0 

0.0 

0.0 

0.0 

0.0 

Algebra 

18a. 

Equations,  1  unknown 

3 

60.0 

0.0 

0.0 

100.0 

18b. 

Equivalent  equations 

2 

100.0 

50.0 

0.0 

50.0 

18c. 

Calculate  power,  sq  root 

7 

58.3 

100.0 

0.0 

0.0 

Vocabul arv 

26a. 

Common  words 

38 

23.8 

42.1 

59.9 

0.0 

26b. 

Aircraft,  tank  words 

1 

16.7 

100.0 

0.0 

0.0 

26c. 

Mean  from  context 

11 

7.0 

36.4 

27.3 

36.4 

26d. 

Contract , abbrev ,  acronym 

113 

72.9 

72.6 

18.6 

8.8 

26e. 

Figurative,  idiomatic 

30 

19.4 

56.7 

13.3 

30.0 

26f . 

Communic,  navig  words 

119 

76.3 

92.4 

-- 

7.6 

26g. 

Rifle,  survival  words 

106 

67.9 

88.7 

-- 

11.3 

Reference  Skills 

27a. 

Locate  documents 

1 

0.8 

100  0 

0.0 

0.0 

27b. 

Locate  &  file 

5 

3.8 

80.0 

20.0 

0.0 

27c. 

Tbl  cont,  index,  gloss 

24 

18.5 

75.0 

12.5 

12.5 

27d. 

Locate  title,  page,  etc. 

6 

4.6 

100.0 

0.0 

0.0 

27e. 

Skim,  scan  for  info 

108 

81.2 

51.9 

21.3 

26.9 

27f . 

Use  cross  references 

40 

31.0 

62.5 

32.5 

5.0 

♦Percents  based  upon  number  failing  pretest. 
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Table  C.l.  (Cont.)  Soldiers  Failing  Pretest:  Percent  Passing  or  Failing 

(On-Line  Lessons  Only)  (N-179) 


Lesson 

Of  Attempting: 
Fail  Pretest 
(No.)  (Pet.) 

Of  Failing:* 
Pass  Short 
(Pet.) 

Pass  Long 
_ (Pet.) 

Not  Pass 
(Pet.) 

Tab! es/Charts 

28a.  Fact  from  2-col  table 

4 

2.8 

100.0 

0.0 

0.0 

28b.  Fact  from  tbl  intersect 

1 

0.8 

100.0 

-- 

0.0 

28c.  Complex  tbl,  cross  ref 

36 

52.2 

41.7 

33.3 

25.0 

28d.  Tbls  locate  malfunction 

55 

38.5 

58.2 

27.3 

14.5 

Illustrations 

29a.  Details  from  ill  us 

22 

16.1 

77.3 

22.7 

0.0 

29b.  Details  from  key,  legend 

2 

1.5 

100.0 

0.0 

0.0 

29c.  Use  cross-sec  view 

1 

0.7 

100.0 

0.0 

0.0 

29d.  3-D  projection 

33 

24.1 

75.8 

-- 

24.2 

29e.  Ulus  to  follow  direct 

1 

0.7 

100.0 

0.0 

0.0 

Flow  Charts 

30a.  Meanings  flow  cht  symbol 

135 

96.4 

99.3 

-- 

0.7 

30b.  Flow  cht,  proced  decis 

4 

50.0 

75.0 

-- 

25.0 

30c.  Ident  organiz  members 

0 

0.0 

0.0 

0.0 

Sgh^matigs 

31a.  Locate  subsys  block 

26 

100.0 

61.5 

26.9 

11.5 

31b.  Components,  signal  path 

21 

100.0 

85.7 

14.3 

0.0 

31c.  Circuit  connects  schema 

12 

66.7 

66.7 

-- 

33.3 

3 Id .  Faulty  compon/trbl shoot 

0 

0.0 

0.0 

0.0 

0.0 

31e.  Symbol:  comp,  signal  path  1 

16.7 

100.0 

0.0 

0.0 

Forms 

32a.  Locate  block  for  info 

1 

0.8 

100.0 

0.0 

0.0 

32b.  Transfer  data 

1 

0.8 

100.0 

0.0 

0.0 

32c.  Enter  info  on  form 

29 

22.5 

58.6 

10.3 

31.0 

32e.  Use  form  to  find  info 

1 

0.8 

100.0 

0.0 

0.0 

Outlinina  (Todic  gr 

34a.  Ident  main  idea 

5 

100.0 

20.0 

-- 

80.0 

34b.  Titles  for  outline 

0 

0.0 

0.0 

-- 

0.0 

34c.  Select  details 

0 

0.0 

0.0 

-- 

0.0 

34d.  Label  topics 

121 

92.4 

50.4 

49.6 

♦Percents  based  upon  number  failing  pretest. 
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Table  C.l.  (Cont.)  Soldiers  Failing  Pretest:  Percent  Passing  or  Failing 

(On-Line  Lessons  Only)  (N=179) 


Of  Attempting:  Of  Failing:* 

Fail  Pretest  Pass  Short  Pass  Long  Not  Pass 


Lesson 

[ran 

(Pet.) 

( Pet . ) 

■EhI 

Editinq 

36a.  Spell  common  words 

19 

40.4 

84.2 

10.5 

5.3 

36b. 

Spell  task  words 

8 

18.2 

50.0 

50.0 

0.0 

36c. 

Capitalization 

33 

94.3 

36.4 

18.2 

45.5 

36d. 

Correct  misspelling 

6 

21.4 

66.7 

33.3 

0.0 

36e. 

Punctuation 

34 

97.1 

26.5 

17.6 

55.9 

36f . 

Grammar  rules 

15 

42.9 

93.3 

6.7 

0.0 

36g. 

Rewrite  paragraph 

2 

66.7 

100.0 

-- 

0.0 

36h. 

Appraise  &  adjust 

10 

35.7 

70.0 

-  - 

30.0 

Precautions 

40a.  Knowledge  prevent  injury 

0 

0.0 

0.0 

0.0 

40b. 

Safety/security  probs 

23 

17.3 

91.3 

-- 

8.7 

40c. 

Emergency  action 

75 

55.6 

81.3 

“  - 

18.7 

Recoanition 

41a.  Label  objects 

2 

1.5 

100.0 

0.0 

0.0 

41b. 

Hand  &  arm  signals 

114 

87.7 

86.8 

12.3 

0.9 

41c. 

Equip  damage,  defects 

31 

24.0 

93.5 

6.5 

0.0 

41d. 

Move,  aline,  connect 

12 

9.3 

83.3 

16.7 

41e. 

Objs  size,  shape,  etc. 

2 

1.5 

100.0 

0.0 

0.0 

41g. 

Choose  action 

35 

27.3 

51.4 

-- 

48.6 

4 1  h . 

Use  symbols  &  codes 

11 

8.6 

72.7 

18.2 

9.1 

♦Percent  based  upon  number  failing  pretest. 
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Gage  Measures  lessons  were  generally  easier.  Although  83  percent  who  attempted 
lesson  5a  on  reading  and  interpreting  gages  failed  the  pretest,  more  than  half  passed  the 
pretests  on  each  of  the  remaining  lessons  in  the  series.  Relatively  few  soldiers  failed  any 
of  these  lessons. 

Results  were  mixed  for  the  Spatial  lessons.  Many  failed  the  pretest  on  manipulating 
objects  to  aline  them  (66%)  or  relating  symbols  to  systems  (53%).  Some  40  percent  did 
not  pass  this  latter  lesson. 

Most  of  those  needing  instruction  on  the  lessons  in  the  Lines  series  were  successful 
after  the  short  lesson.  This  was  not  true  in  the  series  on  Planes.  More  than  one  out  of 
every  five  who  required  instruction  in  lessons  8c,  8d,  or  8e  failed  the  lesson.  Lessons  on 
Angles  and  Triangles  generally  found  soldiers  failing  the  pretests,  although  only  two 
lessons  (Identify  Angles  and  Identify  Altitudes  and  Bisectors)  were  attempted  by  as  many 
as  100  people.  Most,  however,  were  able  to  master  the  content  after  the  short  lesson. 

About  83  percent  of  those  attempting  the  single  lesson  on  Solids  failed  the  pretest. 

Of  that  group,  three-fourths  passed  the  test  after  the  short  lesson. 

In  the  series  on  Addition  and  Subtraction,  soldiers  were  most  likely  to  require 
instruction  on  positive  and  negative  numbers;  performing  functions  with  linear,  dry,  liquid, 
and  degree  measures;  and  estimating  the  sum  or  difference.  They  were  most  likely  to 
remain  unsuccessful  after  instruction  in  the  lessons  on  estimating  the  sum  or  difference; 
24-hour  time;  and  linear,  dry,  liquid  and  degree  measures. 

For  the  Multiplication  and  Division  Series,  initial  difficulties  were  greatest  with 
negative  and  positive  numbers  (71  percent  failed  the  pretest);  problems  with  decimals  in 
the  divisior  or  dividend  (69%);  and  working  with  whole  and  decimal  numbers  (55%).  More 
than  half  of  those  who  completed  the  lesson  on  decimals  in  the  divisor  and  dividend  failed 
it. 

The  series  on  Fractions  and  Decimals  was  also  difficult.  More  than  half  of  the 
soldiers  attempting  the  pretest  failed  it  in  lessons  on  converting  decimals  and  percents  to 
fractions,  multiplying  and  dividing  fractions,  and  estimating  sum,  products  and  quotients. 
More  than  a  third  of  those  taking  the  lessons  on  converting  decimals  to  fractions, 


multiplying  and  dividing  fractions,  and  estimating  sum,  product  and  quotient  could  not  pass 
the  test  after  completing  both  the  short  and  long  lessons. 

In  the  Geometry  series  soldiers  appeared  familiar  with  geometric  shapes  and  their 
names  but  not  with  the  processes  of  computing  areas.  Fewer  than  40  soldiers  attempted 
any  of  the  lessons  15f  through  15j.  Among  this  small  group,  almost  all  failed  the  pretest 
but  passed  the  lesson  after  instruction.  The  exception  was  the  lesson  on  oscilliscope 
readings,  in  which  six  of  the  11  attempting  it  were  not  successful. 

Combinations  of  Processes  were  also  relatively  difficult.  Only  two  of  the  seven 
lessons  addressed  by  soldiers  in  this  series  were  passed  on  the  pretest  by  as  many  as  half 
of  the  students.  Seven  of  the  19  (37%)  talcing  the  lesson  on  using  all  processes  with 
whole  and  mixed  numbers  failed  it;  this  was  true  for  nine  of  the  20  who  took  the  lesson 
on  using  all  processes  with  units  of  measurement.  On  the  other  hand,  success  rates  were 
high  for  the  lessons  on  using  information  from  charts  and  graphs,  and  on  using  all 
processes  with  word  problems. 

Only  two  lessons  from  the  series  on  Graphing  in  the  Coordinate  Plane  were 
attempted  by  soldiers  in  this  evaluation.  More  than  half  failed  the  pretest  on  identifying 
grid  coordinates,  but  almost  all  eventually  succeeded  with  the  lesson.  Only  about  one  in 
five  failed  the  pretest  on  six-digit  coordinates  and  map  line  intersections,  but  of  these  21 
percent  again  failed  to  pass  the  test  after  instruction. 

The  Algebra  lessons  were  attempted  by  few  soldiers.  Those  on  deriving  equations 
were  quite  difficult,  but  the  lesson  on  using  a  calculator  to  find  power  and  square  root 
was  successful  for  all  who  failed  the  pretest.  None  of  the  Trigonometry  lessons  was  used. 

Verbal  lesson  performance.  The  difficulty  and  success  rates  in  the  Vocabulary  series 
varied  from  lesson  to  lesson.  Soldiers  were  most  likely  to  need  instruction  in  lessons  on 
the  subjects  of  contractions,  abbreviations,  and  acronyms;  communication  and  navigation 
words;  and  rifle  or  survival  words.  Only  small  numbers  failed  the  pretests  for  the  lessons 
on  identifying  meaning  from  context  and  understanding  the  meaning  of  figurative  or 
idiomatic  terms,  but  of  this  group  about  three-tenths  did  not  pass  the  test  after 
instruction.  Reference  Skills  appeared  easy  except  for  the  lesson  on  skimming  and 
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scanning,  in  which  81  percent  failed  the  pretest.  Of  those  taking  that  lesson,  27  percent 
failed  the  test  again  after  completing  both  the  short  and  long  lesson  instruction. 

Soldiers  typically  passed  the  pretests  in  Tables  and  Charts  except  for  the  lesson  on 
using  complex  tables  and  cross-references.  Nine  of  the  36  needing  this  lesson  remained 
unsuccessful  after  completing  the  instruction.  Illustrations  also  appeared  to  hold  little 
difficulty.  More  than  75  percent  of  the  soldiers  attempting  the  lessons  in  this  series 
passed  the  pretest  Among  soldiers  taking  the  lessons,  the  only  failures  occurred  in  the 
lesson  on  using  three-dimensional  projection. 

Some  of  the  content  in  the  series  on  Flow  Charts  and  the  one  on  Schematics 
appeared  to  be  initially  unfamiliar.  Almost  all  soldiers  failed  the  pretest,  but  passed  the 
posttest  in  the  lessons  on  identifying  the  meaning  of  flow  chart  symbols;  identifying  and 
locating  subsystems,  and  so  forth;  and  identifying  components  and  signal  paths.  Several 
still  had  problems  after  completing  the  lessons  on  using  flow  charts  for  procedural 
decisions  or  tracing  circuit  connections  in  a  schematic. 

Very  few  soldiers  failed  the  pretest  for  any  of  the  lessons  in  the  Forms  series.  Of 
the  30  percent  who  required  instruction  in  the  lesson  on  entering  selected  information 
onto  a  form,  31  percent  were  not  successful. 

Two  of  the  lessons  in  the  series  on  Outlining  a  Topic  or  Sentence  were  attempted  by 
any  soldier.  All  of  the  five  who  took  the  pretest  on  identifying  the  main  idea  failed  it; 
four  of  them  failed  again  after  instruction.  Some  92  percent  failed  the  pretest  on  using 
numbers  and  letters  to  label  topics,  and  half  of  this  group  failed  the  test  following  the 
lesson. 

Not  all  of  the  lessons  in  the  series  on  Editing  were  this  difficult;  in  only  two, 
identifying  words  needing  capitalization  and  applying  punctuation  rules,  did  appreciable 
numbers  fail  the  test  after  instruction.  Two-thirds  or  more,  however,  who  took  the 
pretest  failed  it  for  the  lessons  on  capitalization,  punctuation,  and  rewriting  a  paragraph. 

Soldiers  generally  knew  the  content  of  the  lessons  in  the  series  on  Precautions,  or 
mastered  it  after  instruction.  In  the  series  on  Recognition,  88  percent  of  those  taking 
the  pretest  for  the  lesson  on  using  and  interpreting  hand  and  arm  signals  required 
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instruction;  virtually  all  passed  the  posttest.  The  one  lesson  in  the  series  that  remained 
difficult  was  that  on  choosing  an  action  by  using  sight,  hearing,  and  touch.  Of  the  35 
who  failed  the  pretest,  almost  half  were  unsuccessful  following  instruction. 

Time  to  Complete  JSEP 

Table  C.2  separates  the  soldiers  attempting  each  on-line  lesson  into  several  groups: 
those  taking  only  the  pretest  (passing  it  and  exempted  from  the  lesson);  those  taking  only 
the  short  lesson  (passing  it  or  failing  it  with  no  long  lesson  available);  and  those  taking 
the  long  lesson  (passing  or  failing).  Paper  lessons  were  excluded  because  no  accurate  time 
figures  were  recorded  for  these.  For  each  group  the  table  reports  the  median  minutes  to 
complete  the  lesson.  The  median  is  the  point  separating  the  group  into  two  halves  (50 
percent  score  above  the  median  and  50  percent  below  it)  and  it  was  selected  for  this 
report  because  it  is  less  sensitive  to  extreme  scores  than  the  mean.  For  example,  half  of 
all  soldiers  took  fewer  than  16  minutes  (the  median  time)  to  complete  Lesson  la.  But  as 
we  saw  in  the  body  of  the  report,  one  soldier  required  221  minutes.  If  that  figure  had 
been  averaged  in  to  produce  the  mean  time  in  minutes,  it  would  have  given  an  unrealistic 
picture  of  how  long  the  typical  soldier  spent  on  Lesson  la.  Table  G2  also  lists  the 
median  time  to  complete  for  soldiers  passing  the  pretest  (13  minutes  for  Lesson  la),  those 
taking  only  the  pretest  and  the  short  lesson  (for  la,  62  minutes),  and  those  requiring  the 
pretest,  short  lesson,  and  long  lesson  (146  minutes). 

The  most  obvious  finding  from  Table  C.l  is  that  there  was  a  great  deal  of  variation 
both  within  and  between  lessons.  JSEP  appears  truly  to  have  been  used  as  a  self-paced 
program.  There  was  so  much  difference  in  the  median  amount  of  time  required  to 
complete  lessons  (for  example,  contrast  Lessons  lc  and  le)  that  it  does  not  make  much 
sense  to  calculate  the  average  time  in  JSEP  when  soldiers  may  have  different 
prescriptions  of  lessons.  Again,  there  was  so  much  variation  in  time  between  soldiers  who 
passed  the  lessons  at  different  stages  that  it  is  difficult  to  say  how  long  it  took  soldiers 
to  complete  any  given  lesson.  A  soldier  who  passed  all  of  the  pretests  in  Numbering  and 
Counting  could  have  completed  that  series  in  an  hour  and  a  half.  A  soldier  who  needed 
to  work  through  each  short  lesson  in  that  series  would  have  required  about  seven  hours 
to  do  so. 
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Table  C.2.  Median  Number  of  Minutes  in  Each  Prerequisite 

Competency  (N-179) 


Soldiers: 


All 

Passing 

Taking 

Taking 

Lesson 

Soldiers 

Pretest 

Short  Only 

Numberina  4  Counting 

la. 

Match  name 

16 

13 

62 

146 

lb. 

Sequence 

55 

15 

49 

no 

lc. 

Before,  between 

175 

14 

75 

202 

Id. 

Greater/lesser 

26 

5 

33 

150 

le. 

Ordinal  position 

9 

8 

29 

70 

If. 

Place  value 

15 

7 

22 

75 

lg. 

Round  whole/decimal 

46 

7 

45 

102 

lh. 

Count  by  1,5,10,  etc. 

30 

9 

34 

-- 

li. 

Match  scale  intervals 

63 

14 

76 

-- 

Linear,  Weight  &  Volume 
Measures 


2a. 

Linear  scale  mark 

7 

7 

24 

62 

2b. 

US  4  metric  measures 

20 

5 

34 

90 

2c. 

Lengths 

8 

7 

21 

108 

2d. 

Weight,  pressure,  torque 

6 

6 

25 

-- 

2e. 

Volume 

17 

11 

46 

2f . 

Non-numeric  calibrated 

6 

5 

16 

-- 

2g. 

Estimate  size,  distance 

45 

8 

36 

68 

Dearee  Measures 

3a. 

Degrees,  mils,  angles, 

temp 

13 

10 

43 

119 

3b. 

Est  angle  <180  deg 

8 

6 

25 

92 

3c. 

Interp  azimuths,  0-6400 

mils 

74 

8 

42 

97 

3d. 

Interp  azimuths,  0-360 

deg 

8 

5 

27 

70 

Time- 

-Tell ina  Measures 

4a. 

Digital ,  analog,  24  hr 

32 

8 

29 

88 

4b. 

Clockface  direction 

11 

5 

16 

32 

4c. 

Est  secs,  mins 

6 

6 

19 

48 

4d. 

Gregorian  4  Julian 

140 

25 

67 

154 

4e. 

Convert  hours,  lOths 

30 

-- 

30 

-- 

4f . 

Convert  to  Zulu 

208 

-- 

-- 

208 

012 


Table  C.2.  (Cont.)  Median  Number  of  Minutes  in  Each  Prerequisite 

Competency  ( N= 179) 


Soldiers : 


Lesson 

All 

Soldiers 

Passing 
_ Pretest _ 

Taking 

Short  Only 

Taking 

long 

Gaae  Measures 

5a.  Read  gage 

31 

10 

31 

88 

5b.  Read  read-out 

6 

5 

14 

42 

5c.  Read  gage,  color 

6 

6 

16 

24 

5d.  Read  scales  (+/-) 

5 

5 

18 

51 

5e.  Read  multiscale 

11 

9 

31 

77 

5f.  Match  gage  to  spec 

10 

9 

25 

69 

5g.  Read  unnumbered/unmarked 

4 

4 

19 

29 

5h.  Read  fluctuating  gage 

24 

14 

34 

89 

5i.  Match  specs/aline 

9 

9 

33 

-  ” 

Soatial 

6a.  Ident  direct  tools  move 

7 

7 

41 

6b.  Manip  to  aline,  etc. 

29 

15 

31 

97 

6c.  2-dimension  to  spatial 

9 

8 

30 

154 

6d.  Symbols  to  systems 

16 

8 

23 

-  - 

lines 

J7a.  Ident  points,  etc. 

14 

6 

18 

40 

7b.  Ident  vert,  horiz,  diag 

11 

7 

25 

62 

7c.  Ident  intersect,  diverg 

5 

3 

16 

28 

7d.  Superimpose  lines 

17 

-  - 

17 

-  - 

PI anes 

8a.  Match  plane  shapes 

5 

4 

16 

8b.  Ident  geometric 

6 

6 

8c.  Apply  shape  terms 

11 

7 

17 

61 

8d.  Match  patterns 

5 

5 

15 

46 

8e.  Figure  orient 

4 

4 

14 

* 

Anales  &  Triangles 

9a.  Ident  angles 

45 

13 

44 

99 

9b.  Ident  types  angles 

37 

12 

38 

9c.  Ident  types  triangles 

19 

5 

23 

9d.  Ident  altitudes,  bisect 

40 

11 

41 

83 

9e.  Name  angles 

21 

8 

30 

Sol  ids 

10a.  Match  names/figures 

14 

4 

14 

41 

Table  C.2.  (Cont.)  Median  Number  of  Minutes  in  Each  Prerequisite 
Competency  ( N= 179) 


Lesson 

All 

Soldiers 

Soldiers: 

Passing 

Pretest 

Taking 

Short  Onlv 

Taking 

Lonq 

Addition  &  Subtraction 

12a.  Whole,  no  carry 

10 

10 

53 

12b. 

Whole,  carry 

7 

7 

28 

164 

12c. 

Decimals,  carry 

8 

8 

29 

97 

1 2d . 

Pos  4  neg  numbers 

56 

7 

31 

154 

12e. 

24-hour  time 

34 

14 

49 

190 

1 2f  - 

Increments  measuring 

15 

13 

47 

121 

12g. 

Linear,  dry,  liquid,  deg 

85 

23 

68 

175 

12h. 

Estimate  sum,  diff 

86 

15 

33 

92 

Multiol ication  &  Division 

13a.  Whole  numbers 

23 

21 

72 

201 

13b. 

Whole  &  decimal 

39 

16 

55 

142 

13c. 

Deci,  divisor,  dividend 

71 

18 

51 

120 

13d. 

Neg  1  pos  numbers 

17 

8 

18 

-- 

13e. 

Est  product,  quotient 

9 

8 

30 

87 

Fractions/Decimal s 

14a.  Est  frac  length,  area, 
vol 

8 

7 

24 

no 

14b. 

Reduce 

11 

9 

34 

147 

14c. 

Convert  decs/fracs 

21 

15 

53 

100 

14d. 

Convert  decimal s/%'s 

34 

14 

47 

162 

14e. 

Add,  subtract  fracs 

18 

10 

31 

147 

14f . 

Multiply,  divide  fracs 

56 

17 

59 

162 

14g. 

Estimate  fraction 

71 

7 

24 

102 

Geometry 

15a.  Draw  plane  fig 

4 

4 

13 

15b. 

Match  figures/names 

6 

5 

21 

55 

15c. 

Label  objects,  figures 

6 

6 

20 

-- 

15d. 

Use  protractor 

4 

4 

31 

88 

15f . 

Area,  perimeter  rectang 

23 

4 

17 

38 

15g. 

Radius,  area,  circum  cir 

29 

18 

29 

119 

15h. 

Measure  rectang  solids 

48 

-- 

28 

63 

15i . 

Geometric  prob  formulas 

34 

14 

51 

222 

15  j . 

Oscilloscope  readouts 

121 

60 

124 
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Table  C.2.  (Cont.)  Median  Number  of  Minutes  in  Each  Prerequisite 
Competency  ( N=1 79) 


Lesson 

All 

Soldiers 

Soldiers : 

Passing 

Pretest 

Taking 

Short  Only 

Taking 

Lonq 

Combination  of  Processes 

16a.  Locate  center  of  object 

3 

3 

16b.  Compute  averages 

16 

12 

35 

70 

16c.  All  proc,  whole/mixed 

173 

11 

39 

232 

16d.  All  proc,  units  meas 

75 

11 

42 

82 

16e.  Info  charts,  graphs,  etc. 

20 

13 

40 

97 

16f.  Conversion  probs 

29 

15 

31 

102 

16g.  Ratio,  proportion  probs 

14 

5 

16 

53 

16h.  All  proc,  word  probs 

18 

8 

21 

47 

GraDhinq  in  the  Coordinate 

PI  ane 

17a.  Grid  coords  map 

26 

11 

35 

125 

17b.  6-digit  coords 

9 

9 

26 

-  - 

17c.  Plot  point,  dist,  dir 

6 

6 

-- 

-- 

Algebra; 

18a.  Equations,  1  unknown 

151 

25 

156 

18b.  Equivalent  equations 

68 

-- 

25 

110 

18c.  Calculate  power,  sq  root 

23 

13 

30 

-- 

Vocabul ary 

26a.  Common  words 

14 

13 

36 

251 

26b.  Aircraft,  tank  words 

7 

7 

51 

-  . 

26c.  Mean  from  context 

6 

5 

23 

150 

26d.  Contract, abbrev,  acronym 

30 

8 

30 

126 

26e.  Figurative,  idiomatic 

7 

6 

23 

88 

26f.  Communic,  navig  words 

44 

7 

49 

-  - 

26g.  Rifle,  survival  words 

22 

6 

25 

-- 

Reference  Skills 

27a.  Locate  documents 

4 

4 

22 

-  - 

27b.  Locate  &  file 

6 

6 

20 

38 

27c.  Tbl  cont,  index,  gloss 

5 

5 

41 

114 

27d.  Locate  title,  page,  etc. 

27e.  Skim,  scan  for  info 

13 

13 

36 

18 

6 

15 

51 

27f.  Use  cross  references 

13 

11 

35 

76 

Tables/Chart? 

28a.  Fact  from  2-col  table 

4 

4 

15 

28b.  Fact  from  tbl  intersect 

4 

4 

25 

28c.  Complex  tbl,  cross  ref 

36 

19 

54 

94 

28d.  Tbl s  locate  malfunction 

15 

11 

27 

116 
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Table  C.2.  (Cont.)  Median  Number  of  Minutes  in  Each  Prerequisite 
Competency  (N=179) 


Lesson 

All 

Soldiers 

Soldiers: 

Passing 

Pretest 

Taking 

Short  Onlv 

Taking 

Long 

111 ustrations 

29a.  Details  from  ill  us 

10 

10 

29 

73 

29b.  Details  from  key,  legend 

6 

6 

22 

29c.  Use  cross-sec  view 

6 

6 

29 

-- 

29d.  3-D  projection 

4 

4 

17 

29e.  Ulus  to  follow  direct 

7 

7 

23 

-- 

Flow  Charts 

30a.  Meanings  flow  cht  symbol 

19 

3 

19 

-  - 

30b.  Flow  cht,  proced  decis 

15 

7 

30 

-  - 

30c.  Ident  organiz  members 

6 

6 

-- 

-- 

Schematics 

31a.  Locate  subsys  block 

31 

-- 

22 

47 

31b.  Components,  signal  path 

53 

-- 

52 

125 

31c.  Circuit  connects  schema 

18 

6 

25 

-  - 

31d.  Faulty  compon/trbl shoot 

7 

7 

-- 

31e.  Symbol:  comp,  signal  path 

6 

5 

23 

-- 

Forms 

32a.  Locate  block  for  info 

4 

4 

15 

-  - 

32b.  Transfer  data 

8 

8 

18 

-  - 

32c.  Enter  info  on  form 

19 

18 

44 

134 

32e.  Use  form  to  find  info 

6 

6 

21 

-- 

Outlining  (ToDic  or  Sentence) 

34a.  Ident  main  idea 

23 

23 

«  - 

34b.  Titles  for  outline 

7 

7 

-  - 

•  - 

34c.  Select  details 

10 

10 

-  - 

34d.  Label  topics 

89 

48 

90 

-- 

5d i t i nq 

36a.  Spell  common  words 

22 

14 

63 

121 

36b.  Spell  task  words 

14 

13 

51 

235 

36c.  Capitalization 

113 

16 

36 

152 

36d.  Correct  misspelling 

13 

12 

54 

108 

36e.  Punctuation 

213 

249 

113 

249 

36f.  Grammar  rules 

15 

10 

54 

160 

36g.  Rewrite  paragraph 

46 

8 

48 

-- 

36h.  Appraise  &  adjust 

12 

8 

38 
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Table  C.2.  (Cont.)  Median  Number  of  Minutes  in  Each  Prerequisite 
Competency  (N=179) 


Soldiers: 


Lesson 

All 

Soldiers 

Passing 

Pretest 

Taking 

Short  Onlv 

Taking 

Long 

Precautions 

40a.  Knowledge  prevent  injury 

5 

5 

-  - 

-  - 

40b.  Safety/security  probs 

6 

5 

19 

-- 

40c.  Emergency  action 

22 

7 

28 

-- 

Recoonition 

41a.  Label  objects 

4 

4 

17 

41b.  Hand  &  arm  signals 

20 

5 

20 

97 

41c.  Equip  damage,  defects 

12 

10 

33 

120 

41d.  Move,  aline,  connect 

3 

3 

11 

•  - 

41e.  Objs  size,  shape,  etc. 

4 

4 

20 

4 lg .  Choose  action 

5 

5 

17 

41h.  Use  symbols  &  codes 

10 

10 

26 

61 
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The  second  conclusion  is  self-evident:  the  lesson  pretest  saved  instructional  time. 
Soldiers  who  passed  the  pretest  did  so  quickly.  Part  of  the  longer  times  shown  for 
soldiers  who  took  the  short  lessons  or  the  short  and  long  lessons  may  reflect  slower 
work,  but  part  is  attributable  to  the  simple  mechanics  of  going  through  the  instructional 
content. 

Instructionally,  the  time  values  agree  with  the  lesson  test  performance  reported  in 
Table  C.l.  The  more  difficult  a  lesson  was  to  pass,  the  longer  it  took  to  complete. 
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APPENDIX  D 


Soldiers’  Responses  and  Comments— 
Post-JSEP  Attitude  Survey 


Table  D.l.  Characteristics  of  Soldiers 
Completing  JSEP  (N-175) 


Item _ No.  Pet. 

Soldier  Characteristic 
Rank 


El 

1 

0.7 

E2 

10 

6.7 

E3 

24 

16.0 

E4 

76 

50.7 

E5 

28 

18.7 

E6 

11 

7.3 

Other,  not  given 

25 

-- 

Career  Mgt.  Field  (10  or  more) 


Combat  Engineering 

10 

5.7 

Field  Artillery 

20 

11.4 

Signal  Operations 

15 

8.6 

Mechanical  Maintenance 

19 

10.9 

Administration 

14 

8.0 

Supply  &  Service 

22 

12.6 

Transportation 

12 

6.9 

Fewer  than  10  in  CMF 

63 

36.0 

Years  in  Duty  MOS 

Less  than  1 

29 

16.6 

1 

46 

26.3 

2-3 

55 

31.4 

4  or  more 

45 

25.7 

Like  Duty  MOS? 

Yes,  want  to  keep 

100 

57.1 

No,  want  to  change 

67 

38.3 

No  response 

8 

4.6 

Native  Language 

English 

163 

93.1 

Other 

12 

6.9 

D-l 


Table  D. 1 . ,  cont. 


Florida  State  University 
Ouestions 

Percent  Responding 
Strongly 

Aoree  Aaree 

Uncertain 

Di saaree 

Strongly 
Di saaree 

Easy  to  learn  to  use 
computer 

61.1 

33.7 

2.3 

2.9 

Using  computer  3  or  4 
hours  too  long 

8.0 

17.7 

13.1 

47.4 

13.7 

Should  use  written 

assignments  with  computer 

15.4 

32.6 

17.7 

20.6 

12.6 

Lessons  will  help  me  under¬ 
stand  publications  I  use 

20.6 

49.1 

18.3 

9.7 

1.7 

Skills  learned  in  JSEP  will 
help  me  advance  in  grade 

18.3 

33.1 

30.9 

10.9 

6.3 

Instructor  should  teach 
lessons,  not  computer 

5.7 

10.9 

28.0 

34.9 

19.4 

Willing  to  take  more  JSEP 
on  duty 

40.6 

31.4 

14.3 

9.1 

4.0 

Willing  to  take  more  JSEP 
off  duty 

28.0 

26.3 

26.3 

9.1 

9.7 

My  commander  would  release 
me  from  duty  for  JSEP 

23.4 

34.3 

31.4 

5.7 

4.0 

JSEP  should  be  offered  by 
Education  Center 

57.1 

32.0 

9.1 

1.7 

-- 

Table  D. 1 . ,  cont. 


Item _ No.  Pet 

Enrollment  Factor 


Why  Enrolled* 


Commander  recommended 

31 

17.7 

Education  Center  recommended 

46 

26.3 

I  wanted  to 

83 

47.4 

Other  reason 

41 

23.4 

How  Long  Enrolled 

Under  4  weeks 

46 

26.3 

4  to  6  weeks 

104 

59.4 

More  than  6  weeks 

22 

12.6 

No  response 

3 

1.7 

Completed  Prescription 

Yes 

118 

67.4 

No 

42 

24.0 

Not  sure 

9 

5.1 

No  response 

6 

3.4 

Enrolled  Hours  Per  Day 

Under  4  hours 

18 

10.3 

4  hours 

138 

78.9 

More  than  4  hours 

19 

10.9 

Enrolled  Days  Per  Week 

Under  5  days 

13 

7.4 

5  days 

151 

86.3 

More  than  5  days 

9 

5.1 

No  response 

2 

1.1 

Primary  Purpose  Enrolled 

Improve  GT 

150 

85.7 

Pass  SQT 

1 

0.6 

Improve  MOS  skills 

1 

0.6 

Qualify  for  new  MOS 

1 

0.6 

Improve  math  skills 

1 

0.6 

Improve  verbal  skills 

1 

0.6 

Prepare  for  GED 

-- 

-- 

Don't  know/not  sure 

Other 

No  response 

20 

11.4 

*Could  check  more  than  one  answer. 
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Table  D.l. ,  cont. 


Item 

No. 

Pet. 

Effect  of  JSEP 

Effect  on  Motivation  to  Learn 

Became  more  motivated 

135 

77.1 

Became  less  motivated 

4 

2.3 

No  effect  on  motivation 

9 

5.1 

Don't  know  what  effect  was 

25 

14.3 

No  response 

2 

1.1 

How  Much  Did  You  Learn  From  JSEP? 

I  learned  a  lot 

94 

53.7 

I  learned  some 

73 

41.7 

I  didn't  learn  much 

4 

2.3 

I  didn't  learn  anything 

Not  sure/don't  know 

2 

1.1 

No  response 

2 

1.1 

Helped  the  Most  With* 

Learning  to  use  computer 

56 

32.0 

Learn/improve  skills  for  job 

36 

20.6 

Improve  math  skills 

122 

69.7 

Improve  reading,  verbal  skills 
Didn't  help  anything  much 

64 

36.6 

5 

2.9 

Other  answer 

11 

6.3 

Effect  on  How  Well  You  Do  Your  Job 

Positive--learned  much  of  use  to 

job 

43 

24.6 

Average--helped  a  little 

73 

41.7 

Negative--bad  effect  on  job 

3 

1.7 

No  effect  on  job  performance 

47 

27.9 

Not  sure/don't  know 

5 

2.9 

No  response 

4 

2.3 

Helped  or  Caused  Problems  in  These* 

Taught  what  I  need  for  job 

21 

12.0 

Increased  self-confidence 

79 

45.1 

Commander  didn't  like  my  time  in 

JSEP 

7 

4.0 

Motivated  me  to  learn  and  improve 

1  skills 

100 

57.1 

Taught  a  lot  about  Army 

31 

17.7 

Held  me  back  from  unit 

7 

4.0 

Helped  me  improve  GT  score 

59 

33.7 

Helped  me  learn  new  MOS 

9 

5.1 

Gave  me  break  from  routine 

75 

42.9 

No  major  benefits 

14 

8.0 

No  major  problems 

55 

31.4 

♦Could  check  more  than  one  answer. 
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Table  D. 1 . ,  cont. 


Item _ 

Attitude  Toward  JSEP 

What  Did  You  Like  About  JSEP?* 

Chance  to  use  computer 

Way  JSEP  explained  things 

Way  JSEP  drew  pictures  and  diagrams 

Way  computer  responded  to  answers 

Things  I  learned  in  lessons 

Chance  to  learn  and  study  by  myself 

Nothing  in  particular 

Other 

What  Did  You  Dislike  About  JSEP?* 

Way  JSEP  explained  things 

Way  computer  responded  to  answers 

Way  JSEP  drew  pictures  and  diagrams 

Way  tests  were  given 

Errors  in  JSEP  lessons 

Problems  with  computer 

Nothing  in  particular 

Other 

How  Did  You  Feel  About  JSEP  at  First? 
Liked  it  a  lot 
Liked  it  a  little 
Didn't  like  it  very  much 
Didn't  like  it  at  all 
No  response 

How  Do  You  Feel  About  JSEP  Now? 

Like  it  a  lot 
Like  it  a  little 
Don't  like  it  very  much 
Don't  like  it  at  all 
No  response 

Did  Any  Thing  Change  Your  Feelings? 

Yes 

No 

No  response 

How  Would  You  Recommend  JSEP  be  Used? 
Replace  all  BSEP  with  JSEP 
Use  JSEP  as  part  of  BSEP 
Don't  use  JSEP  at  all 
Not  sure  or  don't  know 
Other 

No  response 

*Could  check  more  than  one  answer. 
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Table  D.l. ,  cont. 


Very 

Somewhat 

Don't 

Know 

Useful 

Useful 

Useful 

No  Response 

Item 

No. 

Pet. 

No. 

Pet. 

Pet. 

No. 

Pet. 

Ratings  of  Elements 

Test  before  lesson 

Ill 

63.4 

48 

27.4 

7 

4.0 

9 

5.1 

Short  review  lesson 

118 

67.4 

49 

28.0 

2 

1.1 

6 

3.4 

Test  after  short  lesson 

126 

72.0 

38 

21.7 

4 

2.3 

7 

4.0 

Long  lesson 

97 

55.4 

55 

31.4 

12 

6.9 

11 

6.3 

Test  after  long  lesson 

109 

62.3 

49 

28.0 

7 

4.0 

10 

5.7 

Not 

Don' 

t 

Not  Taken/ 

Usefulness  of  Learning** 

Useful 

Useful 

Remember 

No  Response 

Strateqv  Lessons 

No. 

Pet. 

No. 

Pet. 

No. 

Pet. 

No. 

Pet. 

Time  management 

136 

77.7 

13 

7.4 

16 

9.1 

10 

5.7 

Test  taking 

155 

88.6 

8 

4.6 

3 

1.7 

9 

5.1 

Motivational  skills 

114 

65.1 

23 

13.1 

22 

12.6 

17 

9.7 

Reading  comprehension 

132 

75.4 

8 

4.6 

14 

8.0 

21 

12.0 

Problem  solving 

145 

82.9 

9 

5.1 

10 

5.7 

11 

6.3 

Lessons  Assigned  Were;* 

No. 

Pet. 

Too  easy 

13 

7.4 

Just  the  ones  I  needed 

92 

52.6 

Wanted  to  change  but  couldn't 

21 

12.0 

Most  were  not  needed  or 

wanted 

42 

24.0 

Other 

28 

16.0 

Would  You  Like  to  Change  Way 
JSEP  is  set  up? 


Yes 

65 

37.1 

No 

106 

60.6 

No  response 

4 

2.3 

**See  text;  question  appears  to  have  been  misunderstood. 
♦Could  check  more  than  one  response. 
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Table  D. 1 . ,  cont. 


Item 

Percent  Rating: 

Excellent  Good 

Needs 

ImDrovement 

Poor 

No 

ResDonse 

El ement 

Pictures  on  screen 

43.4 

47.4 

6.3 

1.7 

1.1 

Explanation  of  ideas 

45.1 

48.0 

4.6 

1.1 

1.1 

Examples,  practice  questions 

48.0 

44.0 

.6.9 

0.6 

0.6 

Computer  response  to  practice 
questions 

37.1 

42.9 

17.1 

2.3 

0.6 

Level  of  reading  difficulty 

24.6 

61.7 

9.7 

2.3 

1.7 

Way  tests  are  scored 

25.1 

54.9 

14.3 

4.0 

1.7 

Number  of  tests  required 

30.3 

57.7 

9.7 

1.1 

1.1 

Computer  set-up 

38.9 

49.1 

9.1 

0.6 

2.3 

Controlling  screen,  moving 
through  lessons 

42.3 

44.6 

10.9 

1.7 

0.6 

Choice  of  lessons  assigned 

33.7 

45.1 

16.6 

4.0 

0.6 

Length  of  lessons 

26.3 

56.6 

12.0 

2.9 

2.3 

Amount  of  time  in  J SEP 

26.3 

57.7 

12.0 

2.9 

1.1 

Number  of  lessons  assigned 

29.1 

55.4 

11.4 

2.9 

1.1 

Help  from  instructor 

66.3 

30.3 

1.7 

0.6 

1.1 

Military  content  of  lessons 

29.7 

56.6 

6.9 

4.6 

2.3 

Civilian  content  of  lessons 

24.6 

57.7 

12.6 

2.3 

2.9 
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Table  D. 1 . ,  cont. 


Item 

No. 

Pet. 

Difficulty  or  Problems 

In  general,  how  difficult  were  lessons? 

Too  easy 

6 

3.4 

Rather  easy 

30 

17.1 

Just  right 

107 

61.1 

Rather  difficult 

26 

14.9 

Too  difficult 

No  response 

6 

3.4 

What  was  most  difficult  part  of  JSEP?* 

Using  computer 

4 

2.3 

Reading  lesson  material 

8 

4.6 

Understanding  diagrams 

12 

6.9 

Keeping  my  concentration 

67 

38.3 

Test  questions 

11 

6.3 

Nothing  was  difficult 

80 

45.7 

Other 

16 

9.1 

Did  you  have  any  computer  problems?* 

Screen  became  messed  up 

25 

14.3 

Got  stuck  in  a  lesson 

50 

28.6 

It  marked  correct  answer  wrong 

74 

42.3 

Did  not  respond  to  commands 

31 

17.7 

Computer  was  broken 

9 

5.1 

Not  enough  computers 

4 

2.3 

Didn't  have  any  problems 

55 

31.4 

How  often  did  you  go  to  instructor  for  help? 

More  than  3  times  per  session 

5 

2.9 

One  to  3  times  per  session 

29 

16.6 

Once  per  session 

28 

16.0 

Less  than  once  per  session 

53 

30.3 

Rarely  or  never 

57 

32.6 

No  response 

3 

1.7 

What  kind  of  problems  did  you  ask  about?* 

Problems  with  computer 

65 

37.1 

Problems  understanding  lessons 

66 

37.7 

Problems  with  practice  questions 

35 

20.0 

Problems  with  tests 

30 

17.1 

Rarely  or  never  asked  about  problems 

55 

31.4 

Other 

7 

4.0 

*Could  check  more  than  one  response. 


D-8 


Table  D.I.,  cont. 


Item 


No.  Pet. 


Difficulty  or  Problems 


How  did  instructor  help?* 


Told  me  to  keep  trying  or  review 

14 

8.0 

Explained  in  a  different  way 

96 

54.9 

Adjusted  computer 

54 

30.9 

Gave  other  materials  to  read 

16 

9.1 

Did  or  could  not  help 

1 

0.6 

Rarely  or  never  asked  for  help 

49 

28.0 

Other 

8 

4.6 

*Could  check  more  than  one  response. 
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Table  D.I.,  cont. 


Item 

No. 

Pet. 

What  was  best  about  lessons?* 

Explain  concepts  clearly 

98 

56.0 

Show  how  to  do  things  step  by  step 

129 

73.7 

Illustrate  in  practical,  useful  ways 

93 

53.1 

Give  opportunity  to  practice 

94 

53.7 

Teach  things  never  had  chance  to 

learn 

59 

33.7 

Nothing  was  very  helpful 

4 

2.3 

Other 

5 

2.9 

Don 

't  Know/ 

Yes 

No 

No  Response 

No. 

Pet. 

No. 

Pet. 

No. 

Pet. 

Did  JSEP  help  you: 

Learn  at  own  pace 

152 

86.9 

15 

8.6 

8 

4.6 

Practice  until  you  got  it  right 

155 

88.6 

16 

9.1 

4 

2.3 

Be  motivated  to  learn 

121 

69.1 

36 

20.6 

18 

10.3 

Relax,  not  worry  about  mistake 

103 

58.9 

62 

35.4 

10 

5.7 

Discover  how  to  do  things  on  own 
See  material  presented  step  by 

152 

86.9 

18 

10.3 

5 

2.9 

step 

155 

88.6 

13 

7.4 

7 

4.0 

Anything  frustrate  you  or  prevent  learning 

? 

No. 

Pet. 

Illustrations  took  too  long 

54 

30.9 

Lessons  were  boring 

17 

9.7 

Lessons  repeated  things  too  much 

31 

17.7 

Couldn't  work  with  other  soldiers 

3 

1.7 

I  didn't  make  enough  effort 

3 

1.7 

Lessons  were  confusing 

8 

4.6 

Lessons  not  ones  I  needed  or  wanted 

24 

13.7 

Didn't  like  touching  screen 

20 

11.4 

Couldn't  see  what  I  got  wrong  on 

tests 

111 

63.4 

Didn't  like  how  computer  addressed  me 

4 

2.3 

Computer  malfunctioned  too  often 

17 

9.7 

Didn't  have  any  trouble  learning 

42 

24.0 

Other 

15 

8.6 

How  do  you  usually  learn  best? 

Computer  or  teacher  -  no  difference 

14 

8.0 

Computer,  in  general 

29 

16.6 

Teacher,  in  general 

27 

15.4 

Depends  -  sometimes  prefer  one  or 

other 

95 

54.3 

Not  sure 

2 

1.1 

No  response 

8 

4.6 

*Could  check  more  than  one  response. 
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SOLDIER'S  COMMENTS 


POST-JSEP  ATTITUDE  SURVEY 


3 .  How  do  vou  feel  about  vour  current  MOS  assignment? 

Not  MOS  related. 

Lack  of  field  exercise. 

I  like  my  job  but  I  want  to  do  something  different. 

It's  not  challenging  enough  for  me. 

I  like  to  get  into  what  I  do  best. 

I  did  not  get  the  job  I  wanted  when  I  came  in. 

I  don't  even  work  with  my  MOS. 

Because  I  don't  like  it  anymore. 

There  isn't  any  school  associated  with  this  MOS  except 
AIT.  There's  only  two  [places]  that  you  can  go  to — 
state  or  overseas.  Only  one  place  stateside.  Ft. 

Sill. 

Too  many  hours  and  less  harassment. 

a)  Don't  like  field  duty;  b)  This  MOS  not  that  useful 
in  civilian  life?  c)  Like  to  have  a  technology- 
related  MOS. 

When  I  joined  the  Army  they  didn't  explain  to  me  what  my 
job  was. 

I  want  to  raise  my  GT  score. 

No  civilian  occupation. 

It  is  not  what  I  thought  it  would  be. 

I  want  to  be  a  welder  or  a  mechanic. 

Too  many  YEMT. 

It's  for  people  with  no  future I ! I 

Because  of  the  job  relatives  (sic)  or  the  civilian 
techniques. 

I  want  something  different. 

This  MOS  isn't  challenging  enough  for  me. 

It's  not  the  job  I  wanted. 

I  feel  I  could  do  something  better. 

I'm  beginning  to  dislike  working  in  it  very  much;  it 
don't  feel  like  a  career  anymore. 

I  want  to  go  to  school  and  go  through  ROTC  then  proceed 
to  law  school. 

It  is  boring  and  has  no  challenge. 

Other;  I  like  this  job,  but  I  want  to  advance  as  a 
paramedic. 

I  want  to  become  a  law  enforcement  officer. 

But  there  is  a  cut-off  on  promotion  and  advance  school 
is  needed  to  further  my  career. 

I  like  my  primary  MOS  better  and  know  the  job  well. 

I  want  to  go  into  accounting. 

It  is  no  challenge  for  the  civilian  economy. 

Too  stressful;  too  hard  to  get  promotions. 
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I  wanted  95B  before  I  took  11B.  I'm  trying  for  95B. 

The  promotions  are  too  slow. 

I  don't  feel  that  my  time  in  garrison  has  a  useful 
purpose. 

Too  much  dumb  stuff  is  happening,  everything  at  the  last 
minute. 


7 .  Why  did  vou  enroll  in  JSEP? 

Wanted  to  raise  my  GT  scores. 

To  raise  my  GT  scores. 

To  bring  my  GT  score  up. 

I  wanted  to  refresh  myself. 

To  help  my  GT  score. 

To  improve  my  skill  and  GT  score. 

To  bring  my  GT  score  up. 

To  raise  my  GT  score. 

To  raise  my  GT  scores. 

I  needed  to  raise  my  GT  score. 

I  want  to  improve  my  GT  score. 

Help  for  promotion  (looks  good  on  record) . 

To  improve  common  skill. 

I  wanted  to  improve  my  skill  and  GT  scores. 

Improve  GT  for  OCS. 

To  raise  my  GT  score. 

To  improve  my  GT  score. 

I  enroll  in  the  program  because  I  wanted  to  raise  my  GT 
score. 

Improvement  of  my  GT  score  for  LPN  school. 

To  raise  my  scores. 

To  better  my  GT  score. 

Raise  GT  score. 

To  help  myself. 

Improve  GT  scores. 

To  improve  my  GT  score. 


8 .  What  was  vour  primary  purpose  in  taking  JSEP? 

I  wanted  to  improve  GT  scores. 

I  want  to  change  my  MOS  to  a  MOS  without  a  security 
clearance. 

To  improve  my  overall  skills. 

I  thought  I  would  get  promotion  points. 
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10 .  Did  taking  JSEP  have  anv  effect  on  vour  motivation  to 
learn  or  to  improve  vour  skills? 

But  the  machine  worked  too  slow,  so  it  kind  of  brought 
my  motivation  down  at  times. 

It's  a  very  educational  program. 


1 1 .  In  general,  how  difficult  were  the  JSEP  lessons? 

Some  were  easy,  some  were  difficult  and  it  made  you 

stop  and  think.  Some  of  the  questions  were  from  years 
ago  and  were  hard  to  answer  the  questions  before  the 
lesson. 

They  just  make  a  person  think  harder.  It's  not  at  all 
easy. 


12 .  What  did  you  like  about  JSEP? 

To  learn  at  my  own  pace. 

A  chance  to  learn  new  things. 

To  learn  about  my  weak  points,  to  improve. 

How  important  it  was  to  pay  attention  to  instructions. 

When  I  answered  something  wrong,  it  explains  to  me  why 
it  was  wrong  then  how  I  have  to  go  about  correcting 
it. 

Just  the  way  it  was  set  up. 

The  consideration  the  instructor  has  for  the  students. 

The  ability  to  work  at  my  own  pace. 

I  don't  really  know  except  it  made  me  aware  of  my 
weaknesses. 

It  gave  me  insight  on  my  weaknesses  after  being  out  of 
school . 

This  system  should  be  used  everywhere. 

A  chance  to  refresh  my  skills. 

It  spends  too  much  time  drawing  and  it's  very  slow  at 
times  responding  to  the  answer. 

When  taking  a  test  you  could  not  go  back  to  change  your 
answer,  only  if  you  missed  the  key  and  hit  the  wrong 
key. 

The  long  lesson  it  put  you  into  after  you  fail  the  test 
the  second  time. 

After  you  take  the  first  posttest  before  a  lesson,  you 
should  be  able  to  say  "Yes"  or  "No"  to  taking  the  test 
over  before  getting  into  the  lesson.  Sometimes  you 
missed  it  by  one  and  know  the  one  you  missed. 

Some  of  the  material  was  too  long  and  drawn  out,  and  too 
much  detail. 
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At  times  the  explanations  were  unclear  as  for  their 
answers . 

Short  lessons. 

In  a  lot  of  pretest  and  tests  I  would  abbreviate  either 
too  many  or  not  enough  answers. 

I  didn't  like  not  knowing  what  I  missed  on  each  test. 

Sometimes  the  computer  doesn't  explain  how  to  answer 
some  of  its  questions;  your  answer  may  be  correct  but 
the  computer  says  it ' s  wrong . 

The  pictures  and  diagrams  took  too  long  to  draw. 

Not  knowing  exactly  what  questions  were  wrong  on  my 
test. 

Not  being  able  to  review  right  or  wrong  test  answers. 

Sometimes  if  you  had  the  correct  answer  it  would  tell 
you  you  were  wrong  and  type  the  same  answer  you  had 
written  in  the  first  place  where  you  left  off.  You 
had  to  start  that  specific  section  over  again.  It 
wasted  precious  time. 

When  you  sign  off  it  doesn't  take  you  back  exactly  where 
you  stopped. 

Small  problems,  nothing  too  serious  to  handle. 

It  doesn't  explain  or  let  you  know  where  you  made  a 
mistake. 


15 .  What  did  JSEP  help  vou  with  the  most? 

JSEP  gave  me  lots  of  encouragement  to  hang  in  there  to 
finish. 

Learning  how  to  learn;  I  been  away  from  school  for  5 
years. 

It  helped  me  in  my  math  but  I  still  haven't  got  it  quite 
right. 

Helped  me  all  around. 

Helped  me  see  what  understanding  the  problem  does  for 
me .  Thinking . 


16 .  What  was  best  about  the  wav  JSEP  lessons  work? 

To  realize  that  education  is  important. 

Sometimes  they  would  go  into  directions  too  deep;  I 
would  overlook  ones  I  should  have. 

It  was  a  good  refresher. 

Some  things  I've  never  seen  before  JSEP. 
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prevented  vou  from  learning? 

I  lost  interest  with  all  the  extra  drawing  they  do. 

At  times  not  enough  time  given  to  solve  problems; 
therefore  many  mistakes  were  made. 

Lessons  were  too  long. 

Some  of  the  material  was  disgusting,  too  repetitious, 
and  ridiculous. 

I  think  a  person  who  takes  a  test  should  be  allowed  to 
see  what  he  missed. 

Taking  a  test  and  not  putting  in  the  right  word  or 

abbreviation  on  pretest.  It  would  confuse  me  after  I 
started  a  lesson  because  I  answered  the  pretest 
different. 

Paper  lessons.  Going  from  computer  to  paper  didn't  help 
my  concentration. 

Some  of  the  material  wasn't  useful  to  me  and  my  primary 
MOS. 

Sometimes  it's  hard  to  touch  it  exactly;  you  should  be 
able  to  use  the  tab  area  more. 

Like  it  was  said  before,  I  really  wasn't  thinking  about 
what  I  was  doing. 


18 .  What  was  the  most  difficult  part  about  JSEP  for  vou? 

The  ten-second  paragraph  scanning.  Not  enough  time. 
Lines  are  close  together,  big  fingers  tend  to  hit  more 
than  one  line.  Then  no  time  to  change  answer  and 
shift  next. 

Especially  those  in  land  navigation. 

Understanding  abbreviations  for  word.  Sometimes  I  would 
get  so  frustrated  I  would  put  wrong  answers  because  of 
abbreviations . 

Staying  awake,  due  to  working  hours. 

The  paper  lessons  took  time. 

Math  word  problems,  but  I  will  have  it  done  before  I  get 
out  complete  JSEP. 

After  looking  at  the  screen  for  long  periods  of  time. 

My  eyes  got  tired  after  staring  at  the  computer  for  too 
long. 

Negative  numbers  and  algebra  should  be  explained  more. 

I  thought  the  lessons  were  very  useful  to  me. 

Some  was  difficult  but  understanding  from  many  points  of 
view  was  the  key. 
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19 .  Please  tell  us  how  useful  the  learning  strategies 

lessons  were  bv  circling  the  letter  that  best  matches 

vour  opinion. 

I  like  the  motivational  skills;  it  made  me  look  forward 
to  passing  all  my  tests  the  first  time. 

Could  have  used  more  in  problem  solving.  Reading 
comprehension  could  have  expanded  more  in  detail  on 
finding  the  topic  matter. 

Reading  and  comprehension  questions  were  confusing. 

They  didn't  seem  to  match  the  answers  given.  (Some  of 
the  questions) . 

I  felt  some  lessons  were  too  easy,  but  overall  it  was  a 
great  chance  to  refresh  and  learn  more. 

They  are  set  up  very  good. 

Overall  they  helped  in  a  positive  way. 

We  should  be  able  to  see  and  use  our  test  results  to 
better  ourselves. 

There  were  limited  explanations  when  the  computer  showed 
me  what  had  been  wrong. 

I  think  the  lessons  were  very  useful.  The  lessons  were 
written  good. 

Should  have  more  problem  solving  dealing  with  math 
problems,  because  this  is  all  that  is  on  the  TABE 
tests. 

I'd  like  to  take  some  reading  comprehension  because  I 
feel  I'm  weak  and  need  strengthening. 


21.  What  do  you  think  about  the  lessons  assigned  to  vou  in 

JSEP? 

The  lessons  were  not  all  easy,  at  times  some  of  the 

material  presented  I  had  never  seen  before.  Therefore 
it  helped  me  a  lot. 

Other  lessons  should  be  chosen  for  one's  MOS. 

Not  finished  all  the  way  with  JSEP.  So  I  won't  answer. 

Somt  of  the  lessons  I  needed,  some  I  didn't. 

Most  of  the  time. 

Some  lessons  were  the  ones  I  needed  to  take  but  some 
were  useless  to  me. 

I  guess  they  were  good  enough. 

Simply  because  I  learned  something  new  with  each  lesson. 

I  really  needed  the  math  lessons,  but  some  of  the  others 
are  too  easy. 

They  were  the  ones  I  wanted  to  learn. 

They  gave  me  a  lot  of  good  methods  I  had  forgotten  and 
enhanced  my  knowledge  to  learn  other  calculations. 

Need  more  problem  solving. 

Everything  fell  into  place,  day  by  day. 
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25 .  What  kinds  of  problems  did  you  usually  ask  vour 

instructor  about? 

Problems  with  tests;  sometimes  the  answer  would  not  be 
accepted. 

Paper-based  lessons. 

What  the  computer  wanted. 

She  assisted  some  on  the  paper-based  lessons. 

Was  always  there  when  I  needed  a  helping  hand  and  kept 
encouraging  me  when  I  was  down  to  drive  on. 

The  instructor  was  very  helpful  and  always  monitoring  my 
progress. 

Gave  good  step-by-step  instruction  along  with  the 
computer. 

Also  explain  things  in  the  computer  way. 


29 .  How  would  vou  recommend  JSEP  be  used? 

Both  programs,  teachers  and  computers  should  be  used  for 
a  clearer  view. 

I  feel  good  about  JSEP.  It's  self  paced.  Has  some 
problems  that  will  be  worked  out  as  you'll  learn  and 
do. 

The  computer  is  fine  but  sometimes  a  teacher  can  do  a 
better  job  (explaining,  that  is) . 

A  teacher  is  very  valuable  to  explain  when  confused. 

BSEP  is  a  waste  of  time  and  money.  You  can't  learn 
things  you  forgot  in  2  weeks. 

I've  never  attended  BSEP  but  heard  it  was  boring. 


32.  Was  there  anything  that  made  vour  feelings  change? 

Using  the  computer  more. 

It  gave  me  a  chance  to  feel  my  way  and  understand  the 
best  method  for  the  test  at  hand. 

I  learned  some  skills  and  refreshed  some  old  skills. 

Because  when  I  first  started  JSEP  I  didn't  understand 
the  class  but  now  I  do. 

After  I  started  working  the  lesson,  I  got  involved.  The 
more  I  got  involved  the  more  I  wanted  to  do. 

Therefore  I  spent  as  much  time  as  possible  working  on 
JSEP. 

Boring  lessons. 

Yes,  I  know  that  in  the  future  I  will  need  to  refresh 
myself  and  I  hope  JSEP  is  there. 

Some  material  was  too  repetitious  and  long  and 

ridiculous  and  made  me  feel  stupid,  or  insulted  my 
intelligence. 
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Pushing  buttons  for  4  hours  5  days  a  week. 

Some  of  the  tests  in  the  lesson. 

Sometimes  I  would  become  bored,  disgusted  at  messing  up 
or  forgetting  how  to  do  a  problem  or  problems. 

When  I  began  to  notice  an  improvement  in  my  weaker 
subject. 

I  have  never  been  to  BSEP. 

Refresher  course. 

The  lessons  got  very  difficult  toward  the  end. 

Hand  and  hand  with  the  instructor. 

The  math  word  problems  (smile) 

To  complete  it. 

The  long  hours  in  front  of  the  computer  is  tiring  on  the 
eyes;  I  missed  the  classroom  conversations. 

Too  many  questions  on  common  tasks  and  not  enough  on 
math  skills. 

Stay  with  teachers;  computers  just  don't  cut  it. 

It  refreshed  my  memory. 

I  enjoyed  coming  here  first,  then  to  work. 

You  don't  use  many  of  the  keys  on  the  computer. 

Looking  at  the  screen  for  an  hour  is  tiring  and  the 
lessons  get  boring. 

After  a  few  days  with  JSEP  I  understood  its  purpose. 

The  use  of  the  computer  not  let  you  go  on  until  it  sure 
you  could  pass  the  test  on  different  subjects. 


33 .  In  what  wavs  has  taking  JSEP  helped  you  or  caused  you 

problems? 

Learn  more  on  common  tasks. 

Overall  it  was  a  good  course.  I  would  recommend  it  to 
others . 

Makes  me  learn  on  my  own  pace. 

I  feel  JSEP  has  great  intention  and  teaches  lots.  If  I 
was  ever  confused  it  was  because  of  the  computer,  more 
probably. 

Overall,  JSEP  helped  me  more  than  caused  me  any 

problems.  It  refreshed  my  mind  on  different  things. 

I  liked  the  course. 

It  was  a  major  problem  as  far  as  time  consuming.  My 
supervisors  nagged  me  every  day  about  when  I  would 
finish.  Yet,  I  was  told  this  is  a  self-paced  program. 

Enabled  me  to  use  the  computers. 

When  I  take  the  test  to  improve  my  GT  I  will  know  how 
much  I  comprehend. 

It  helped  me  review  a  lot  of  things  that  I  was  not  clear 
on. 

I've  enjoyed  the  session  a  lot;  it  may  help  me  dearly  in 
my  military  career. 
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Would 

up? 


s  set 


Go  deeper  into  lessons  than  just  the  basic  everything. 

More  math  and  vocabulary  lessons. 

More  job  skills. 

When  taking  the  test,  let  us  know  what  we  did  wrong  so 
we  don't  keep  making  the  same  mistakes.  Plus  there  is 
too  much  waiting  around  for  the  drawing  to  get  done. 

I  think  that  a  short  review  before  a  test  should  be 
offered  for  those  who  are  not  familiar  with  a  certain 
lesson. 

Change  the  ten-second  paragraph  scan  to  at  least  15 
seconds  or  morel 

By  adding  more  lessons  dealing  with  vocabulary  and 
reading  comprehension. 

Enrolled  learn  BSEP  as  is  and  throw  away  JSEP. 

Some  of  the  lessons  I  think  it  should  concentrate  on 
ASVAB  lessons  only  and  a  few  on  common  tasks. 

Eliminate  the  material  that  doesn't  refer  to  one's  MOS 
and  eliminate  all  ridiculous  lessons  (such  as  lessons 
that  aggravate  you  making  you  do  the  whole  lesson  over 
again  when  it's  .  .  . 

I  like  to  have  some  class  lessons  with  the  teacher. 

For  people  to  take  it  every  other  day. 

The  way  the  tests  are  given;  the  pretest  should  be  given 
after  a  slight  review. 

I  would  break  the  program  by  half  teaching  with  the 
computer  and  half  with  the  instructor. 

I  would  say  the  tests  in  general  need  to  be  worked  on 
some  more;  other  than  that,  it  was  OK. 

I  would  give  the  soldier  more  of  what  he/she  needs  from 
the  JSEP. 

Give  examples,  or  lessons,  before  pretest.  One  word  can 
change  a  correct  answer. 

I  wish  the  computer  would  tell  me  what  my  test  mistakes 
were. 

The  way  each  lesson  has  only  a  certain  number  of 

exercises.  I  would  make  it  so  that  if  an  individual 
wanted  to  get  more  practice  he  could  go  back  to  a 
specific  lesson  and  work  on  weaknesses. 

Students  at  the  end  of  each  lesson  or  test  should  be 
shown  what  they  missed  and  how  to  do  it  right.  This 
way  they  will  know  and  learn  more  in  the  end. 

I  would  let  the  computer  show  what  answers  are  wrong  on 
the  test,  and  give  an  illustration  on  how  it  wants 
some  of  its  questions  answered  on  what  form. 

If  you  are  off  duty  you  should  be  able  to  wear  what  you 
want. 

The  way  that  the  final  outcome  of  the  answers  are  given. 
Just  let  the  soldiers  know  what  questions  were  wrong 
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because  they  may  come  across  it  again,  and  if  they 
don't  know  it  was  wrong  the  first  time  they  will  get 
it  wrong  again. 

Let  us  see  test  results  on  screen  and  give  a  review 
whether  we  passed  the  test  or  not. 

I  would  have  classes  in  the  morning  and  afternoon;  in 
other  words,  classes  all  days  (one  student)  morning 
and  afternoon. 

The  computer  lesson,  the  ones  which  are  difficult  and 
try  to  make  them  easier  to  understand. 

Add  it  to  the  BSEP  program. 

A  person  should  be  allowed  as  much  time  in  the  other 
portion  of  JSEP.  Because  it  is  most  helpful.  I  would 
like  to  say  I  enjoyed  the  program  but  my  supervisor 
made  it  very  hard  on  me  as  far  as  adoption  in  the 
program,  they  want  me  in  and  out.  But  I  was 
interested  in  learning  and  doing  well  on  my  tests. 

They  almost  drove  me  into  getting  out  after  2  months. 

We  should  be  able  to  see  our  JSEP  test  results  to  better 
ourselves.  If  we  knew  what  we  got  wrong  then  we  could 
be  100%  next  time  we  see  the  problem.  This  is  with 
pretests. 

Make  fewer  unnecessary  graphics. 

More  on  math  skills,  less  on  common  task  questions. 

I  would  incorporate  some  classroom  instruction  as  well 
as  computer  lessons. 

Have  class  with  teachers,  not  computers.  They  can't 
answer  as  well. 

I  think  JSEP  should  be  set  up  so  that  most  of  the 

material  is  dealing  with  the  GT  part,  and  the  rest  on 
your  job  skills. 

When  the  computer  is  drawing  and  you  have  already 

finished  reading  the  paragraph  you  have  to  wait  until 
the  picture  is  done. 

Have  headsets  and  computer  material  synchronized  to 
explain  problems  and  tests.  Let  problems  missed  be 
seen. 

Have  more  paper  tests  in  math  and  vocabulary  at  the 

beginning  since  most  people  take  JSEP  to  improve  their 
GT  score. 

When  you  retest,  retest  only  what  you  didn't  understand; 
not  the  whole  thing  from  scratch. 

More  hours. 

Explain  lesson  more  clearly. 


3 5 .  Please  rate  the  following  JSEP  elements.  Circle  the 
letter  which  best  represents  vour  opinion. 

Get  away  from  the  basics.  Test  the  knowledge  of  a 
soldier  then  go  from  there. 
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The  program  needs  to  be  set  more  toward  the  MOS  for  that 
soldier  than  what  it  is.  Because  his  or  her  MOS  can 
be  a  life-giving  or  -shaking  thing. 

The  comment  that  I  gave  on  the  last  part  should  be 
enough. 

To  be  able  to  go  back  in  a  test  and  learn  what  you  had 
done  wrong.  (Attention,  or  didn't  know  proper  way) 

Just  eliminate  all  unnecessary  material,  (lessons  and 
questions) . 

Look  at  page  10,  #34. 

More  specific  information  on  paper-based  lessons. 

The  computer  could  respond  a  little  faster. 

Extend  it  a  little  longer  so  we  could  finish  all  our 
lessons. 

Don't  treat  us  as  stupid. 

I  would  let  the  JSEP  class  be  taught  by  a  teacher 
because  the  computer  gets  boring. 

The  word  math  problems  seem  hard  but  that's  because  I 
haven't  had  them  in  a  long  time  (hope  smile) 

No  comment. 

The  program  is  good  to  refresh  skills  that  we  don't  use 
in  our  military  jobs,  but  need  for  advanced  education. 

They  were  pretty  much  OK;  not  much  to  say. 

It  should  be  a  little  more  difficult.  Most  of  this 
stuff  is  basic  and  we  should  already  know  this. 

JSEP  is  not  just  for  soldiers;  it's  good  for  civilians 
also.  It's  mostly  basic  that  refreshes  your  memory. 

Encourage  people  with  low  GT  scores  to  take  it. 

Give  more  variety  like  food  service  classes,  mail  clerk 
classes,  club  management  weekend  classes,  night 
classes,  etc. 

Show  your  mistakes  on  the  test. 

Everything  was  fantastic;  I  had  no  problems  in  it  and  I 
think  it  ought  to  stay. 

More  computers,  and  more  time. 


36 .  Additional  comments. 

I  feel  that  the  more  the  program  is  geared  towards  the 
MOS  of  soldiers  the  more  beneficial  it  will  be  as  far 
as  a  learning  tool,  because  a  soldier  is  interested  in 
his  job  and  not  the  surrounding  MOSs. 

The  computer  is  very  helpful  in  the  area  that  I  have 
studied  in. 

Most  of  the  time  everything  is  well  explained. 

Sometimes  you're  not  sure  how  they  want  a  question 
answered  and  you  get  it  wrong  when  you  knew  the  right 
answer. 

I  have  seen  an  improvement  in  my  job  performance  since  I 
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began  JSEP.  I  feel  that  this  is  a  motivational  skill 
for  all  soldiers. 

Even  though  these  classes  are  time  consuming,  I  feel 
that  everyone  should  have  the  opportunity  to 
experience  the  classes.  The  knowledge  that  this  class 
has  provided  for  me  is  immeasurable  and  will  prove 
beneficial  as  I  progress  further  in  my  field.  I  have 
enjoyed  being  a  part  of  this  course  and  feel  it  should 
be  offered  post-wide. 

I  think  the  instructors  are  great.  These  two  have  been 
supportive  and  understanding  of  me  from  the  beginning. 
They  both  show  professionalism  and  self-esteem  that 
makes  them  outstanding  instructors.  Thanks,  Guysi 

I  think  it's  a  good  program.  The  computer  has  a  few 
errors.  They  [need]  to  concentrate  more  on  ASVAB 
testing. 

Although  I  may  disagree  with  JSEP  in  some  areas,  I  still 
think  it's  a  good  program  but  I  am  uncertain  about  the 
effectiveness  of  material.  However,  I  was  able  to 
learn  some  important  information  that  could  help  me  in 
life,  and  I  am  grateful. 

I  feel  overall  JSEP  is  headed  in  a  great  direction. 
Helping  people  who  want  to  be  helped. 

I  liked  the  JSEP  program  from  an  overall  viewpoint. 

Some  of  the  lessons  were  what  I  needed  but  some  were 
useless  because  I  already  know  them  or  it  didn't  have 
anything  to  do  with  some  of  the  things  I  needed  to 
know. 

It  is  a  very  good  system.  I  would  recommend  it  to 
everybody. 

No  comments.  I  thank  you,  too! 

Even  though  JSEP  was  long  and  some  material  could  be 
eliminated  for  [it  is]  too  time  consuming.  I  think- 
JSEP  is  one  of  the  best  programs  the  Army  has  come  out 
with.  If  a  computer  can  teach  a  slow  learning  person 
like  me,  the  Army  has  really  outdone  itself. 

Students  should  be  able  to  ask  and  answer  the  computer's 
questions.  This  idea  if  put  into  effect  will  make  the 
JSEP  class  perfect,  besides  the  computers  problems. 

JSEP  courses  were  really  helpful  for  me  to  get  more 
knowledge  of  military  education.  However,  the  main 
purpose  I  had  was  increasing  my  GT  scores  so  that  I 
had  been  bored  while  I  was  in  JSEP. 

I  feel  JSEP  is  a  good  program.  But  when  your  math  skill 
needs  improvement,  more  of  this  should  be  emphasized. 
Common  tasks  are  good,  but  does  not  have  much  to  do 
with  the  GT  test. 

My  instructor  was  very  pleasant  and  helpful  during  the 
entire  enrollment  period.  The  JSEP  program  has  some 
kinks  that  need  to  be  improved,  but  overall  I  like  it. 

I  think  people  should  teach  people  and  leave  computers 
to  the  scientists. 
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Get  rid  of  BSEP. 

When  I  had  BSEP,  20  people  were  in  the  class.  Three  out 
of  the  20  people  advanced.  The  rest  of  us  didn't  go 
on  because  we  didn't  score  high  enough.  It's  hard  to 
jump  into  something  you  haven't  really  thought  about 
for  a  long  time,  and  accomplish  it  in  2  weeks. 

This  has  been  a  pleasure  for  my  learning  experience  and 
would  love  to  do  more  with  this  computer. 
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APPENDIX  E 


Instructors’  Responses  and  Comments 
JSEP  Instructor  Survey 


Tab's  E.l. 

Responses  to  JSEP  Instructors'  Survey 

(N=l 1 ) 

Item 

No. 

Pet. 

Teacher  Characteristic 


Highest  Degree  Completed 


j  r 

Bachelors 

8 

73 

Masters 

3 

27 

Have  Studied 

Adult  education 

6 

55 

Educational  foundations 

8 

73 

Teaching  methodology 

8 

73 

Special  education 

4 

36 

Had  Prior  Experience 

Teaching  ABE/GED 

7 

64 

Teaching  with  computer-based  instruction 

4 

36 

Using  computers  in  general 

6 

55 

Teaching  enlisted  personnel 

11 

100 

Have  Taught  BSEP  or  JSEP  before 

No 

3 

27 

Yes 

8 

73 

If  yes,  for 

1  year  or  less 

1 

9 

2  years 

3 

27 

3  or  more  years 

3 

27 

Have  Taught  JSEP  for 

Less  than  6  months 

3 

27 

6  months  to  1  year 

6 

55 

More  than  1  year 

2 

18 

Received  training  from 

FSU  staff  person 

5 

45 

JSEP  instructor/education  staff 

4 

36 

No  training 

2 

18 
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Tab! e  E.  1 . ,  cont. 


How  Important?*  Covered? 

Item _ Important  Some  Not _ Yes  No 

Importance  of 
Training  Area 


Operating  the  computer 

100 

80 

20 

Getting  in  and  out  of  JSEP 

91 

9 

-- 

80 

20 

Registering  a  soldier 

91 

9 

-- 

80 

20 

Dealing  with  computer  problems 

91 

9 

-- 

80 

20 

Helping  soldiers  with  learning 
di fficul ties 

64 

36 

-- 

44 

56 

Training  other  instructors 

55 

45 

-- 

40 

60 

What  to  do  -  soldiers  stuck  in 
lesson 

100 

-  - 

80 

20 

Managing  JSEP  classroom 

70 

30 

-- 

70 

30 

Operating  learning  strategies 
module 

40 

60 

-- 

60 

40 

♦Percentages  based  on  number  responding  to  item. 
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Table  E.I.,  cont. 


Item 

Percent  Rating  Preparation:* 
Very  Good  Fair  Poor/None 

Not  AddIv 

Preparation  in  Training  Area 

Operating  the  computer 

82 

9 

9 

-- 

Getting  in  and  out  of  JSEP 

82 

9 

9 

-- 

Registering  a  soldier 

73 

18 

9 

-- 

Dealing  with  computer  problems 

27 

64 

9 

-- 

Helping  soldiers  with  learning 
difficulties 

64 

18 

9 

9 

Training  other  instructors 

27 

18 

18 

36 

What  to  do  -  soldier  stuck  in 
lesson 

60 

30 

10 

Managing  JSEP  classroom 

91 

-- 

9 

-- 

Operating  learning  strategies 
module 

27 

45 

27 

-- 

♦Percentage  based  on  number  responding  to 

item. 

Item 

Percent 
Had  this 
Yes 

Reporting: 

resource 

No 

Helpful 
Yes  No 

Resource  Availability 

Telephone  contact  -  JSEP  resource 

peopl e 

91 

9 

82 

Computer  contact  -  JSEP  resource  | 

peopl e 

55 

45 

55  27 

Training  workshop 

18 

82 

55 

Meetings  or  conferences 

9 

91 

45  9 

Instructor's  manuals 

91 

9 

73  9 

Other  (ot;er  instructors) 

-- 

9 

9 
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Table  E. 1. ,  cont. 


Percent  of  Time  Spent: 

Item _ Average  Highest  Lowest 

Task 

Showing  soldiers  how  to  use 


program  and  hardware 

13.2 

25.0 

5.0 

Circulating  and  observing  soldiers 

27.8 

50.0 

10.0 

Helping  with  program  or  hardware 

problems 

9.6 

15.0 

2.0 

Record  keeping  and  administrative 

18.9 

50.0 

0.0 

Using  supplemental  materials  with 

soldiers 

5.6 

20.0 

1.0 

Answering  soldiers'  questions  about 

lessons 

17.2 

30.0 

5.0 

Other  (write  in) 

6.7 

40.0 

0.0 

Paper  lessons  (1) 

40.0 

Briefing  students'  supervisors  (2) 

5.0 

Counseling  (1) 

15.0 

Computer  maintenance  (1) 

5.0 

Tabl e  E. 1 . ,  cont. 


Item 

Percent  Rating  As: 

Excellent  Good 

Needs  Some 
Imorovement 

Poor 

Can't 

Judqe 

Aspect  of  JSEP 

On-line  instructional  program 
(lessons) 

18 

27 

45 

9 

Paper-based  lessons 

-- 

45 

18 

27 

9 

Soldier  management  system 

36 

36 

9 

9 

9 

Hardware  (computers  and  other 
equipment) 

18 

55 

18 

9 

-- 

Classroom  setting  (room,  light, 
set  up) 

9 

27 

36 

27 

-- 

Manual s 

45 

55 

-- 

9 

-- 

JSEP  training 

18 

45 

9 

9 

9 

How  best  use  JSEP  (%) 

JSEP  replace  BSEP  18 

JSEP  as  part  of  BSEP  64 

Do  not  use  JSEP 
Not  sure 

Other  (use  for  MOS  and  CTT 
training;  combine  with  teacher 
and  supplemental  lessons;  add 
traditional  instruction  and 
replace  BSEP;  add  short,  hands- 
on  test-taking  class)  27 
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Additional  Instructors’  Comments 

The  instructors  were  asked  to  add  any  comments  about  JSEP  they  pleased  to  the 
written  survey,  and  most  did  so.  One  mstructor  simply  wrote,  "my  students  enjoy  JSEP." 
Others  were  more  descriptive.  One  stated: 

I  have  taught  several  classes  from  seventh  grade  through  adult  and  I  find  that  the 
Job  Skills  Education  Program  works  best  in  terms  of  interest  and  retention.  It  does 
not  appear  to  be  a  chore  to  the  student.  I  also  find,  from  having  taught,  that  the 
best  way  to  learn  is  to  teach.  JSEP  gives  the  student  an  opportunity  to  "teach" 
himself,  thus  he  gets  more  from  the  lesson. 

Another  was  understanding  about  problems,  writing  that  "I  believe  JSEP  has  a  good 
future.  Of  course,  since  it  is  so  new,  improvements  need  to  be  made,  but  this  is  to  be 
expected."  Other  evaluations  were: 

I  feel  JSEP  is  a  good  program.  It  will  help  the  soldiers  improve  in  areas  such  as 
reading  and  grammar,  which  are  very  difficult  to  do,  as  well  as  in  the  SQT.  It 
should  be  given  a  better  reputation  and  a  broader  use  than  it  has  now.  I  believe  it 
can  be  a  very  valuable  and  effective  learning  tool  for  the  Army. 

A  writer  who  identified  herself  as  a  substitute  instructor  said: 

JSEP  has  proven  itself  effective  on  this  post  in  large  part  due  to  a  very 
conscientious  instructor.  As  JSEP  is  further  developed,  I  feel  it  important  to 
continue  to  provide  good  instructor  training  and  to  maintain  a  good  student- 
instructor  ratio  so  that  an  instructor  can  continue  to  monitor  closely  individuals’ 
problems  and  progress. 

Another  combined  praise  with  a  recommendation. 

Most  JSEP  lessons  are  excellent.  At  least  90  percent  of  our  students  prefer  JSEP  to 
classroom  instruction.  I  think  it  is  a  great  program  that  will  be  even  better  when 
the  final  lesson  revisions  are  complete.  Paper  lesson  19b  needs  to  be  eliminated 
completely.  It  is  too  difficult  for  the  students.  None  of  the  students  have  ever 


been  exposed  to  logarithms  and  they  will  never  use  what  they  learn.  It  is  very 
frustrating  for  the  students. 

Three  other  writers  included  broader  recommendations. 

"JSEP  is  a  very  good  idea;  however,  the  current  system  does  nothing  within  the 
instructional  area.  Students  are  so-called  "pre-tested"  rather  than  receiving  a 
diagnostic  test  which  determines  weak  and  lacking  areas  that  are  "addressed"  by  both 
instruction  and  remedial  training.  The  paper  lessons  are  not  of  any  true  value  to 
the  soldier’s  learning  or  training  program.  The  paper  lessons  should  be  designed  to 
teach  the  student,  and  not  "reprints"  of  information  or  documents  available  through 
Army  manuals  and  publications." 

"100%  of  the  soldiers  that  walk  in  my  door  want  GT  improvement,  as  well  as  their 
commanders.  It  is  my  opinion  that  although  "JSEP  was  not  written  to  improve  GT 
scores,"  as  I  have  been  told,  a  needs  analysis  should  have  revealed  this  as  a  crucial 
part  of  the  Army’s  program  in  basic  academic  skills  training.  If  JSEP  is  a 
replacement  program  for  BSEP,  then  it  should  meet  the  needs  of  these  soldiers.  If  it 
is  supplemental  to  BSEP,  then  how  will  the  Army  educational  system  ever  provide 
standardized,  quality  instruction  when  BSEP  will  still  differ  from  post  to  post? 

Beyond  this  problem,  I  am  satisfied  with  the  design  of  the  curriculum  of  JSEP.  The 
self-paced,  individualized  instruction  allows  soldiers  to  gain  confidence  in  skills  they 
thought  they  were  incapable  of  doing  before.  I  also  believe  it  truly  helps  them 
improve  both  study  and  test-taking  skills,  in  and  out  of  JSEP.  The  management 
system  is  another  plus  for  JSEP.  It’s  great  to  have  the  grades  and  progress  recorded 
automatically  --  if  only  I  could  keep  attendance  records  on  it,  too!  Certain  options 
on  the  menu  are  still  unavailable;  however,  I  hope  this  will  be  remedied  soon.  As  a 
teacher  I  miss  the  interaction  with  the  student  that  I  had  with  BSEP.  It  is  much 
more  difficult  for  me  to  follow  their  progress  and  diagnose  any  deficiencies.  I  also 
find  myself  feeling  guilty  about  not  knowing  how  they’re  doing  or  where  they  need 
help." 

"Change  the  questions  in  this  survey.  This  survey  suggests  that  JSEP  is  a  purely 
academic  educational  tool,  which  is  incorrect.  Over  40  percent  of  my  students  are 
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enrolled  in  their  MOS  or  team  training.  JSEP  has  great  potential  in  teaching 
soldiers  their  MOS  skills  in  addition  to  GT  improvement  educational  skills." 
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